sidecrew

You ask for a change to your source code, and you get it — made on your Mac, by models that cost you no Claude tokens, and only after a machine has checked it.

Claude Opus plans the work and reviews what came back. Small models running locally (MLX, no network) do the narrow parts. Between them sits a gate a machine can run — for a code change: the diff stayed where it was asked to, tsc is clean, and every test that passed before still passes. Claude never sees anything that did not survive it.

$ sidecrew fix change_plan.json
fix 2026-09-20T09-14-02Z-rename: 1 step, 12 tasks, 13.0 GB free ÷ 6.5 GB per slot = 2; 1 worker up
step 1/1 "rename the helper": capturing the baseline
rename-01: survived (src/util/asset.util.ts)
rename-02: survived (src/util/asset.util.ts, src/util/asset.util.spec.ts)
…
multi-06: failed at compile · escalating — renamed the symbol 7 times over, 5 compile errors
11/12 survived · 0 → 0 tsc errors · .sidecrew/runs/2026-09-20T09-14-02Z-rename/result.json

Illustrative shape, not a captured transcript — it is the real output format with the counts from the React measurement below, and no client's file names. 11 of 12 with a 95 % interval of [0.615, 0.998] is the number; the line above is what it looks like while it runs.

Unit tests are workload #1 — the same shape with a mutation-kill gate instead — and the numbers for both are below, each with its interval.

sidecrew needs 24 GB of installed RAM. Below that a 7B cannot sit beside a normal working set (measured), so sidecrew refuses to run and tells you why rather than quietly becoming a different tool (ADR-0073). sidecrew doctor answers this before you plan anything.

And the other half of the ledger, measured 19–20 Sep 2026: the workers are free, the coordination is not. Opus planning costs 8,400–22,100 tokens per task, depending almost entirely on how many tasks are in the plan — see What this costs to run below. A page that says "the work is free" without saying that is telling you half of it.

Before any number below: how tightly to read it

Every rate on this page is an interval, never a point, and the reason is measured. Replaying a frontier model's own diffs — which the gate finds no defect in — through the gate a second time, it disagreed with itself on 2 of 19 candidates: D = 0.105, 95 % interval [0.013, 0.331] (experiments/gate-error-rate/, against a rule frozen before the first replay). That rule's own verdict at D > 0.10 is UNACCEPTABLE, and its consequence is this paragraph: no survival rate here is published as a bare fraction, D travels beside each of them, and no two of our rates are compared without asking whether their difference survives it.

What that does and does not damage. D counts the gate refusing changes that were fine. Nothing in it is evidence of the gate admitting something bad. So:

The gate admits nothing that fails — stands. The gate refuses only things that fail — measured false, at roughly one evaluation in ten, and unproven at any tighter bound.

Two of the three known causes are now recorded on every verdict — memory pressure (ADR-0066) and a baseline captured on a different calendar day (ADR-0069). The third is undiagnosed, and both measured disagreements happened on a quiet machine, so it is neither of the first two. D was measured on workload #2a's gate; it says nothing about workload #1's mutation gate, which is a different oracle (§4.4 of the rule).

Workload #2a — behaviour-preserving changes, measured

Against a decision rule frozen before the tool had produced a single verdict (experiments/go-no-go-2a/results/REPORT.md). S is survival, A is a blind reviewer's approval of the diff. Every cell is k/n with its exact 95 % interval; D = 0.105 [0.013, 0.331] applies to every S.

Read the intervals, not the fractions: at these sample sizes 12/12 and 11/12 are not distinguishable, and neither is the 7B from the control on A. Both real projects passed the frozen rule by a margin of exactly zero, on a sample of ten. The survival rate decided nothing — five of six measurable cells sit at or above 0.917, so the rule's ratio clauses were inert, and the only thing that separated a local 7B from a network model was the approval rate.

What that approval rate is about, measured: the local model makes unrequested cosmetic edits — a deleted docblock, a reworded comment, a stray blank line — in 4 of 23 [0.050, 0.388] sampled survivors, against the control's 0 of 23 [0.000, 0.148]. confined ∧ compiles ∧ tests pass cannot see any of it, which is the point.

What you gain over just asking Opus

Measured 18–19 Sep 2026 against the thing a colleague actually does today — asking a frontier model directly, one task at a time, with no plan and no gate. Same 19 hand-written tasks, same two codebases, four arms.

The token columns are counts and are exact. The survival rows are rates, so they carry their intervals, and D = 0.105 [0.013, 0.331] applies to both of them.

A fourth arm ran the same gate over Opus's own diffs, to separate the gate's contribution from the worker's: 19/19 [0.824, 1.000] on the Nest codebase and 18/19 [0.740, 0.999] on the React one. Two things follow, and the second is the more useful.

The gate found nothing wrong with a frontier model's work. Its single rejection there is a tsc fragility it rejects from both arms — so on these tasks the gate adds no safety on top of Opus. Its entire value is that it makes the free worker usable, not that it second-guesses the expensive one.

And it is what separates a worker defect from a project defect. Of the four React failures, three vanish when Opus writes the diff — genuine worker defects — and one reproduces exactly, because the 7B's output for that task was byte-identical to Opus's. Worker-attributable survival is therefore 15/18 [0.586, 0.964], not 15/19. Without that control the small model would have been blamed for a third more failures than it caused — and note that the two intervals overlap almost entirely, so the correction changes the attribution rather than the number.

Three things that buys you, and one it costs.

1. The work is free. Not cheaper — zero worker tokens. Opus plans and reviews survivors; the editing happens on your Mac.
2. On work a compiler and a suite can check, you give up nothing for it. Eighteen of nineteen diffs were byte-identical to what Opus produced. Not "comparable quality" — the same file.
3. When it is wrong, you get told, loudly, and nothing is applied. The four React failures were a symbol renamed seven times over (5 compile errors), a rename that also hit a type, a module path and a public property key (122 compile errors), a change that renamed the file it was given, and one that a frontier model produced identically. Each printed as escalated, each with the exact errors, none merged. Asking Opus directly gives you a diff and the sentence "nothing else changed" — accurate here, and unverified.
4. It costs wall-clock. ~25 s per task interactively with Opus; 216–271 s with sidecrew (15 s to generate, 200–255 s for the project's own suite and tsc). It is free and unattended, not fast.

What is not claimed. The frozen rule's verdict is WITHHELD: every task in these runs is a rename, and the rule refuses a verdict until at least half are null guards, API migrations or dead-code removal. The measurement vindicated that clause rather than surviving it — two frontier models and a 7B emitted identical bytes on 18 of 19 Nest tasks, so that task set cannot rank anything. experiments/status-quo/ has the arms, the protocol frozen before they ran, and the defects.

Requirements