qikly

New: an agent Skill. qikly --install-skill teaches Claude Code, Gemini CLI, Codex or Cursor how to drive qikly. What the Skill contains.

The problem: Your AI writes both the code and its tests. How do you know the tests are really valid?

Who it is for: a developer or team pointing an AI coding agent at a self-contained Python module that transforms data, for example an ETL step, a merge, a calculation or a validation routine, who does not want to trust a green suite when the same agent wrote both the code and the tests. It suits one module at a time in small to mid-sized repositories: when a test fails, only the files that failure names are loaded, so runs stay small and quick. It fits most naturally where verification already has to be independent, such as automotive, medical devices, fintech and defence: ADAS_HEADWAY, a bundled example, checks following distance from forward-radar samples. See What it is for.

Start here

pip install qikly

Four ways in. Pick the row that matches what you have, and ignore the rest of this page until it has run.

--demo works in a throwaway demo/<timestamp>/ folder it expects you to delete. It is for watching, not for building in. --example and --scaffold create a real project in the directory you are standing in, and those are the two to start from.

Then five steps from your module to a first run.

The idea

Imagine a student who writes the exam paper, writes the answer key, and then sits the exam. They pass, and nobody would accept that as evidence they know the material. That is what happens when one model gets a specification containing the acceptance criteria and writes both the code and the suite that checks it: everything goes green, and the green means nothing.

qikly takes the answer key away from the student. It generates a test suite from the acceptance criteria, then writes an implementation and repairs it against that suite until every test passes or a retry budget runs out, recording every failure, every piece of reasoning and every diff.

The part that makes the result mean something: the coding agent never sees acceptance_criteria. It gets the specification with that section stripped out, the same vague brief a developer works from, while test generation gets it in full. When a test fails, the agent sees pytest's output for that test and never the acceptance criteria. Without that asymmetry both sides read the same spec identically and every test passes first try, which proves nothing.

Purple is what the coding agent can see. Teal is what the standard is written from. They never touch. A run that never converges is still worth having: it exits non-zero, names the blocking tests, and keeps the same complete record. The purple arrows are the repair loop, and that is where almost all of a run happens: a failing suite sends the agent the failure text and nothing else, it produces a FIX and a PATCH, and the suite runs again, until the stage passes or the retry budget runs out. It never sees the criteria themselves, so it cannot write code shaped to a bar it was handed. tests/test_withholding.py fails the build if any call site lets one through.

A run works through three stages, integration then system then unit:

1. Integration and system tests are generated first, from the spec alone, before any code exists. They cannot see an implementation because there is not one yet.
2. The coding agent writes the implementation, from the spec minus the criteria.
3. pytest runs. On failure the model produces a FIX (failure summary, root cause, plan, and the files it intends to touch) and then a PATCH (a unified diff of only those files), applied all or nothing. Repeat until the stage passes or the budget is spent.
4. Unit tests are generated last, once real code exists for them to name. This is the only stage allowed to see the implementation.
5. Clearing a stage re-runs the earlier ones, so a later fix cannot silently break something that already passed.

Every arrow back into FIX carries the pytest error text and nothing else, which means the failing tests and not the specification they came from. Unit tests come last because they are the only ones that need to name real functions, which makes them the only stage allowed to read the implementation. Re-running the earlier stages after each success is what stops a later repair quietly breaking something that already passed.

Test generation sees the requirements, the input and output contract, and every acceptance criterion in full. It writes integration, system and unit tests against the standard.

The coding agent sees the same specification with the criteria section removed, plus the text of whatever test just failed. The same vague brief a developer usually works from.

Two ways to get a test suite, and what each can prove

Both are useful and they answer different questions. qikly is not purely one or the other: integration and system tests are written from the criteria before any code exists, the unit stage is written last from the code that just passed them, and --refine-criteria reads a converged implementation to propose criteria the first draft missed. Each of those reads the code on purpose, and none of them can question it.

What makes this different