release-gate

The pre-deploy release gate for AI agents. It renders an evidence-based PROMOTE / HOLD / BLOCK verdict — catching the agent-layer risks that SAST, guardrails, and evaluators structurally miss.

v0.10.1 — the release where the gate stopped being static-only. RG-PII-001: sensitive context reaching the model unmasked on one path while an equivalent path redacts it — reported only on divergence, so your repo supplies its own oracle, a project that masks centrally stays silent, and the rule is structurally unable to punish the fix it recommends. Platform evidence ingestion (release-gate ingest, and now verify --trace too): Langfuse / OpenTelemetry / Arize-Phoenix / Promptfoo exports convert in place, so both the pre-deploy score and a running loop can be gated on telemetry you already emit — no bespoke file, no new instrumentation, no LLM in the loop, so runtime verdicts stay as reproducible as static ones. The loop check now keys on tool name + arguments, so multi-query retrieval is no longer mistaken for a stuck agent and an agent oscillating between two tools now is. A project's own model wrapper counts as LLM usage, so a production LangGraph app whose only model call goes through a local httpx helper is no longer waved through as "not a deployed agent". 93-case benchmark at 100% precision / 100% recall.

v0.9.4 — a lean, three-dependency CLI (pip install release-gate no longer pulls a web/SaaS stack) and a reproducible 93-case benchmark that covers every rule (≥2 vulnerable + ≥2 clean look-alikes each), so the zero-false-positive claim can be checked, not just read. Both sit on top of the v0.9.0 agent-safety catalog (9 new rules + 2 precision upgrades), holding the precision bar at 0 false positives on that labeled benchmark and a framework dogfood (llama_index / crewAI / langgraph / open-interpreter): indirect prompt injection from RAG/tool/HTTP provenance (RG-PROMPT-002), model-driven SSRF / filesystem / SQL sinks (RG-ACTION-002/003/004), secret/PII → prompt data-egress to the provider (RG-SECRET-002, an agent-aware egress path conventional SAST lacks context to model), taint-aware deserialization (RG-EXEC-004), unvalidated model-output parses (RG-PARSE-001), and tool blast-radius + irreversibility gates (RG-TOOL-001 / RG-GATE-001) — plus confirmed taint through the canonical resp.choices[0].message.content extraction and a reproducible PR-gate demo. See the catalog below. Builds on 0.8.5's release-gate pr, the AI-change review gate: one PROMOTE/HOLD/BLOCK on what a pull request introduced (net-new agent risk + lockfile/behaviour drift), plus a GitHub Action command: pr; 0.8.4's security-hardened MCP server (pip install 'release-gate[mcp]'); and 0.8.0–0.8.2's AST-based evidence-citing analysis, deserialization calibration, and team-adoption workflow (--mode / --baseline / --pr-comment).

Why it's not SonarQube: a SAST tool sees eval(x) and asks "is x tainted by SQL/HTTP?" — it has no concept of "x is the model's reply." That blind spot is the entire agent layer: eval/pickle of model output (the CVE-2025-51472 RCE class), user input reaching a system prompt, LLM loops with no cost ceiling. Guardrails filter one input; evaluators score one output; neither blocks a release. release-gate is the gate.

Try it in 30 seconds

pip install release-gate

# ── The wedge: gate a pull request on what IT introduced ──
# One PROMOTE / HOLD / BLOCK on net-new agent risk only (inherited debt is
# shown, never gated) + prompt/model drift. Runs in CI on the PR branch.
git checkout my-feature-branch
release-gate pr --base origin/main
release-gate pr --base origin/main --comment   # GitHub-ready PR comment

# ── Or audit a whole repo (the broader lens) ──
release-gate audit . --mode ci                                    # your repo, in CI
release-gate audit https://github.com/org/repo --mode public-advisory  # any public repo, advisory

Lean by design. pip install release-gate pulls three small, well-audited libraries — pyyaml, jsonschema, cryptography — and nothing else. No web framework, no database driver, no auth stack in the CLI's dependency tree. The release-gate.com server stack is an opt-in extra (pip install 'release-gate[api]'), and the MCP server is another ('release-gate[mcp]').

Output:

  Repo    https://github.com/your-org/your-ai-agent
  Agents  OpenAI / Agents SDK (4 files), LangChain (12 files)

  Readiness Score   42 / 100   ████░░░░░░

  Agent Code Safety  28/100  BLOCK   4 high · 18 med · 0 low
     Driving the score: Dangerous execution sink ×4; LLM call with no token ceiling ×18
  Governance         50/100  Partial   4/8 safeguards declared

  Decision:  ✗  BLOCK

Two axes, on purpose:

• Agent Code Safety — an objective score from the code itself: prompt-injection surfaces, exec/shell sinks fed by model output, LLM calls with no token ceiling, hardcoded keys. It moves per repo and doesn't depend on adopting anything. These are the agent-layer risks generic SAST/SonarQube don't model — release-gate is the layer on top, not a replacement.
• Governance — maturity of your declared, enforceable safeguards (budget ceiling, kill switch, owner, evals, trace policy…). Low here means undeclared, not unsafe.

Run --full for the per-finding breakdown, or scaffold a ready-to-commit governance config from the scan:

release-gate audit . --emit-config -o governance.yaml
# Fill in the TODO lines, then gate every deploy:
release-gate score governance.yaml

What is release-gate?

release-gate sits between your tests and your deployment. It scans your agent code for the failure modes that only exist once an LLM is in the loop, runs evals, validates execution traces, checks cost budgets — then gives you two honest scores and one decision: PROMOTE / HOLD / BLOCK.

SonarQube checks your code. release-gate checks whether your agent change meets its release policy. They're complementary — keep your SAST suite; release-gate covers the agent layer it was never built to see (prompt-injection surfaces, cost-runaway loops, missing kill switches).

$ release-gate score governance.yaml --evals evals.yaml

  release-gate  |  Readiness Scorer  v0.10.1

  Project          customer-support-agent  v1.0.0
  Checks run       5  (5 pass, 0 warn, 0 fail)
  Evals run        7  (7 pass, 0 fail)  pass rate 100%
  Traces checked   1  (0 violations)

  Score            94 / 100   confidence: high

  Dimension Breakdown:
    safety          100  ██████████  (wt 30%)
    cost             90  █████████░  (wt 20%)
    access_control  100  ██████████  (wt 20%)
    fallback        100  ██████████  (wt 15%)
    eval_quality     85  ████████░░  (wt 10%)
    observability    80  ████████░░  (wt 5%)

  Critical failures  none

  Decision:  ✓  PROMOTE  (score 94/100)  exit 0

What it detects — the agent-safety rule catalog

Every finding carries a stable, citable rule id (RG-EXEC-001) that never changes when we reword a title, a one-line rationale, and a mapping to the frameworks you already answer to (OWASP LLM Top 10, NIST AI RMF). "Why did this block my release?" resolves to a rule, not a code dive. Full catalog with fixes and compliance tags: docs/RULES.md.

Two disciplines run through every rule:

• Precision over recall — we don't cry wolf. When the analyzer can't prove a real risk it stays quiet. Zero false positives on the current labeled benchmark and framework dogfood set — the 93-case corpus now carries ≥2 vulnerable and ≥2 clean look-alikes for every rule (including the v0.9.0 catalog), so that result is reproducible per rule (run python benchmark/run.py), not just asserted; and the engine stayed correctly silent across a framework dogfood (llama_index, crewAI, langgraph, open-interpreter).

• Three evidence tiers — a HIGH is watertight or it isn't a HIGH. The tier is decided by provenance we can point at, never by how a variable is spelled. CI can gate on confirmed-only.

  Tier	Max severity	What it requires
  confirmed	high	A traced origin visible in the file, with a citable chain
  inferred	medium	Real dangerous structure, origin guessed from a name — "confirm the source"
  heuristic	low	Pattern present in agent code; no flow established