# Pytest Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/pytest
> Markdown URL: https://aitinkerers.org/technologies/pytest.md
> Technology record last updated: 2026-02-22T16:49:01Z
> Generated: 2026-09-21T05:36:57Z

Pytest is the leading Python testing framework: it scales from simple unit tests to complex functional testing with minimal boilerplate.

Pytest is a powerful, feature-rich Python testing framework, simplifying test creation and execution. It significantly reduces boilerplate compared to the built-in `unittest` module. Key features include automatic test discovery, powerful assert rewriting that provides detailed failure output (no need for `self.assert*` calls), and a modular fixture system for managing test dependencies and state. Use the `@pytest.mark.parametrize` decorator to run a single test function with multiple input/output scenarios, drastically cutting down on redundant code. The framework’s extensive plugin ecosystem further extends its capabilities, supporting everything from parallel test execution (`pytest-xdist`) to code coverage reporting (`pytest-cov`).

- Official technology site: https://pytest.org
- Public AI Tinkerers demos and talks: 6
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Hive Mind vs Solo Agent: A Live Race to See If Multi-Agent Coordination Actually Beats Working Alone](https://montreal.aitinkerers.org/talks/rsvp_ytH2ELCp8bo)

A live head-to-head race pitting a single Claude Code agent against a ruflo Hive Mind, a "Tactical Queen" coordinating 4 specialized workers (architect, coder, tester, reviewer), both solving the same Python coding challenge in real time. The challenge is to build a sliding window rate limiter from scratch, including implementation, tests, and 80%+ coverage. A custom real-time scoreboard dashboard (Node.js + SSE) watches both workspaces simultaneously, streaming logs and tracking progress as files appear, tests run, and coverage is measured. When both sides finish, an automated evaluation scores them on a 100-point rubric (implementation, test pass rate, coverage, edge cases, time). It's a controlled experiment to answer the question: does multi-agent swarm coordination actually produce better code faster than a single focused agent?

- Event context: AI Tinkerers Montreal - April Demo Night — 2026-04-22 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_ytH2ELCp8bo

### [SQAAILab - Software Quality Assurance & Artificial Intelligence Laboratory. (This is pronounced SKYLAB - first American space station in 1973 :) )](https://montreal.aitinkerers.org/talks/rsvp_y9vu-klDzzM)

1. I will briefly explain my goal with this laboratory. 2. Then demonstrate, through some QUICK concrete examples, how I proceed. 2.1 Prompting (RCTFET Style) 2.2 Vibe Coding 2.3 Doc Generation 2.4 Analysis Assistance 2.5 Test Generation

- Event context: AI Tinkerers Montreal - February 2026 Meetup — 2026-02-24 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_y9vu-klDzzM

### [Building Working Code Live: Documentation-First AI Development](https://seattle.aitinkerers.org/talks/rsvp_ctw_B2iINIQ)

I'll demonstrate the spawn-experiments methodology - a systematic approach to AI-assisted code generation using documentation-first prompting with Claude models (Sonnet 4 and Opus) through Claude Code's Task tool. Live demo: building a functional CLI tool from scratch in real-time, showing how structured specifications generate better AI outputs than vague prompts. Based on spawn-experiments research comparing 4 AI development methodologies across multiple experiments, findings show methodology significantly impacts AI development patterns and outcomes, with trade-offs between development speed and code quality. You'll see the actual "code-like prompting" techniques that treat prompts as executable specifications, parallel code generation across multiple threads, and comprehensive quality control including syntax validation, automated test generation, and real-time error handling.

- Event context: AI Tinkerers Seattle September Meetup — September 30, 2025 — 2025-10-01 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_ctw_B2iINIQ

### [Using evals to build robust chatbot assistants](https://dubai.aitinkerers.org/talks/rsvp_VES7t3DOtpk)

I'll be demonstrating a project I am working on where we are using evals to build robust LLM pipelines. Evals help us be more confident shipping LLM based apps to production. I am building a chatbot assistant that is functioning as a Natural Language to SQL agent, allowing users to interact with data in the database, and to get insights from the companies data. This is an actual live project for a business in the UAE The demo will be focusing on how we built evaluation sets to help write deterministic tests for the agent, to give us more confidence in its performance. We will discuss - how we break down the potential query space of the assistant into defined categories using dimensions and error analysis. This helps structure the set of queries we want our bot to work for, as oppose to having an infinite query space which is hard to design good products around. This gives us structure for designing a well performing assistant - we will show how we have written automated evaluation tests, where we pass queries to our bot, check the results it gets from the db, and we validate that against expected data. This is basically writing deterministic unit tests for our non-deterministic agent pipeline

- Event context: AI Tinkerers Dubai Meetup — August 2025 Demo Day — 2025-08-23 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_VES7t3DOtpk

### [Agent testing with transactions](https://seattle.aitinkerers.org/talks/rsvp_l7Lv36QkvdA)

As the agent logic and graph becomes more complicated, the need of establish a stable baseline to iteratively add capabilities and improve quality of agents is critical to those who are like us at Otto building AI agent to do transactions. We take a similar approach as the one proposed in the τ -bench in this paper (https://arxiv.org/pdf/2406.12045) that’s probabilistic oriented rather than deterministic oriented, yet focusing on testing our own scenarios rather than a benchmark of general capability. Would like to share some early learnings and thoughts around stability, setup and point of testing after testing a handful of choices (Pytest vs Puppeteer/Selenium, Github Action vs CircleCI vs Azure DevOps, end-to-end testing vs langGraph node testing).

- Event context: AI Tinkerers Seattle - December 2024 Meetup — 2024-12-12 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_l7Lv36QkvdA

### [AIEval - another testing framework for AI apps](https://nyc.aitinkerers.org/talks/rsvp_N6an7mhmE3o)

a simple decorator/class based expansion on pytest for making better AI app tests

- Event context: AI Tinkerers NYC May 2024 Meetup — 2024-05-22 — New York City
- Public talk page: https://nyc.aitinkerers.org/talks/rsvp_N6an7mhmE3o

## Related Technologies

- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [Claude Opus](https://aitinkerers.org/technologies/claude-opus) ([Markdown](https://aitinkerers.org/technologies/claude-opus.md)) — 18 public demos
- [Flask](https://aitinkerers.org/technologies/flask) ([Markdown](https://aitinkerers.org/technologies/flask.md)) — 24 public demos
- [AIEval](https://aitinkerers.org/technologies/aieval) ([Markdown](https://aitinkerers.org/technologies/aieval.md)) — 1 public demo
- [Anthropic Bedrock](https://aitinkerers.org/technologies/anthropic-bedrock) ([Markdown](https://aitinkerers.org/technologies/anthropic-bedrock.md)) — 1 public demo
- [ChatGPT](https://aitinkerers.org/technologies/chatgpt) ([Markdown](https://aitinkerers.org/technologies/chatgpt.md)) — 83 public demos
- [CircleCI](https://aitinkerers.org/technologies/circleci) ([Markdown](https://aitinkerers.org/technologies/circleci.md)) — 2 public demos
- [Claude](https://aitinkerers.org/technologies/claude) ([Markdown](https://aitinkerers.org/technologies/claude.md)) — 173 public demos
- [Claude Code CLI](https://aitinkerers.org/technologies/claude-code-cli) ([Markdown](https://aitinkerers.org/technologies/claude-code-cli.md)) — 5 public demos
- [Claude Sonnet 4](https://aitinkerers.org/technologies/claude-sonnet-4) ([Markdown](https://aitinkerers.org/technologies/claude-sonnet-4.md)) — 18 public demos
- [GitHub Actions](https://aitinkerers.org/technologies/github-actions) ([Markdown](https://aitinkerers.org/technologies/github-actions.md)) — 20 public demos
- [Node](https://aitinkerers.org/technologies/node) ([Markdown](https://aitinkerers.org/technologies/node.md)) — 99 public demos
- [Puppeteer](https://aitinkerers.org/technologies/puppeteer) ([Markdown](https://aitinkerers.org/technologies/puppeteer.md)) — 7 public demos
- [ruflo hive-mind](https://aitinkerers.org/technologies/ruflo-hive-mind) ([Markdown](https://aitinkerers.org/technologies/ruflo-hive-mind.md)) — 1 public demo
- [Selenium](https://aitinkerers.org/technologies/selenium) ([Markdown](https://aitinkerers.org/technologies/selenium.md)) — 7 public demos
- [Vanna AI](https://aitinkerers.org/technologies/vanna-ai) ([Markdown](https://aitinkerers.org/technologies/vanna-ai.md)) — 2 public demos