ShimGuard

Verify that a GitHub issue closed as "fixed" actually has a merged fix, before you trust the tracker.

npx shimguard-cli verify sybil-solutions/codex-shim --issues 38,41,42,43,45,46

That single command against the real sybil-solutions/codex-shim repo (1,000+ stars) turns up 6 MISMATCH results: 6 security issues, each closed with a "Fixed in PR #52" comment, where PR #52 was never actually merged. The vulnerable code is still in main today. Nobody reading the closed issues would know.

Contents

• Why this exists
• Install
• Quickstart
• CLI reference
• Library API
• How it compares
• What is ShimGuard, and why does it exist
• FAQ
• Contributing
• License

Why this exists

Reading an issue tracker, you trust two signals: the issue's state (open or closed) and the maintainer's closing comment ("fixed in #N"). Neither signal is verified against reality by GitHub itself. A maintainer can close an issue citing a PR that never merged, an automated bot can close on a "fixes #N" keyword in a PR description before that PR lands, or a fix can get reverted after the issue was already closed. Any of these leaves a tracker saying "fixed" about a bug that is still live.

ShimGuard checks the one thing a human skimming issues does not: does the PR the tracker cites as the fix actually show merged: true? It is a small, mechanical, unambiguous check, not a heuristic or a guess.

Install

ShimGuard ships two independent, equally first-class packages -- pick whichever fits your toolchain, or install both. Neither is deprecated in favor of the other; both implement the same "closed issue cites Fixed in PR #N, is #N actually merged" check against the same GitHub REST API.

# npm -- JavaScript/TypeScript CLI + library
npm install -g shimguard-cli
# or run it once with no install
npx shimguard-cli verify <owner>/<repo> --issues <numbers>

# PyPI -- Python CLI + library (genuine port, not a wrapper around the Node binary)
pip install shimguard-cli

The npm CLI requires Node.js 18 or later (uses the built-in fetch API). The Python package's CLI entry point is also shimguard (e.g. shimguard verify sybil-solutions/codex-shim --issues 45,46); see python/README.md and docs/getting-started.md for the Python-specific walkthrough, and CHANGELOG.md for each distribution's version history.

Quickstart

shimguard verify sybil-solutions/codex-shim --issues 38,41,42,43,45,46

ShimGuard v0.1 -- Tracker Verification: sybil-solutions/codex-shim

[MISMATCH] Issue #45 "_resolve_api_key silently falls back to Cursor API key for any model with an empty api_key, forwarding it to arbitrary upstream URLs"
  https://github.com/sybil-solutions/codex-shim/issues/45
  Cited fix: PR #52 (open, not merged)
  Issue is closed and cites PR #52 as the fix, but that PR is open and was never merged.

Summary: 6 MISMATCH, 0 MATCH, 0 UNVERIFIED (6 checked)

Exit code is 1 when any MISMATCH is found (useful for gating CI), 0 when every checked issue's claimed fix actually merged, 2 on a usage or network error.

ShimGuard is not just a gotcha finder: run it against a mix of issues and it correctly reports MATCH, UNVERIFIED, and MISMATCH side by side, exiting 0 whenever nothing is provably broken -- exactly the signal a CI gate needs.

Optional: verify the code, not just the merge status

For an even stronger check, point ShimGuard at the specific file and pattern an issue named as the vulnerable code:

cat > patterns.json <<'EOF'
{
  "45": { "path": "codex_shim/settings.py", "pattern": "cursor_key_fallback" }
}
EOF

shimguard verify sybil-solutions/codex-shim --issues 45 --patterns patterns.json

If the PR is merged but the cited pattern is still present in the file at HEAD, ShimGuard still reports MISMATCH: a merged PR does not guarantee the specific vulnerable line was actually removed.

[!WARNING] pattern is compiled as a JavaScript RegExp, and path is validated to stay within the target repo (no .. traversal to a different repo or API endpoint). Only point --patterns at files you wrote or reviewed yourself. See SECURITY.md.

CLI reference

Usage: shimguard [options] [command]

Verify that GitHub issues closed as "fixed" actually have a merged fix. Catches
security issues marked fixed whose PR was never merged.

Options:
  -V, --version            output the version number
  -h, --help               display help for command

Commands:
  verify [options] <repo>  Check whether closed issues in a repo actually have
                           a merged fix
  help [command]           display help for command

Usage: shimguard verify [options] <repo>

Check whether closed issues in a repo actually have a merged fix

Arguments:
  repo                target repo as <owner>/<repo>, e.g.
                      sybil-solutions/codex-shim

Options:
  --issues <numbers>  comma-separated issue numbers to check, e.g. 38,41,42
  --patterns <file>   JSON file mapping issue number -> {path, pattern} for an
                      optional code-pattern check
  --token <token>     GitHub token for higher API rate limits (defaults to
                      $GITHUB_TOKEN)
  --format <format>   output format: text or json (default: "text")
  -h, --help          display help for command

--format json output is stable and designed for scripts and AI agents to parse directly:

{
  "repo": "sybil-solutions/codex-shim",
  "checked": 1,
  "summary": { "mismatch": 1, "match": 0, "unverified": 0 },
  "results": [
    {
      "issue": { "number": 45, "title": "...", "state": "closed", "htmlUrl": "..." },
      "citedPullRequest": { "number": 52, "state": "open", "merged": false, "htmlUrl": "..." },
      "patternCheck": null,
      "verdict": "MISMATCH",
      "reason": "Issue is closed and cites PR #52 as the fix, but that PR is open and was never merged."
    }
  ]
}

MCP Server

ShimGuard ships a Model Context Protocol server, so an MCP-compatible agent (Claude Desktop, Claude Code, Cursor, or any other MCP client) can call it as a tool instead of shelling out to the CLI and parsing text. It's part of the Python distribution:

pip install "shimguard-cli[mcp]"

Register it with an MCP client such as Claude Desktop:

{
  "mcpServers": {
    "shimguard": {
      "command": "shimguard-mcp"
    }
  }
}

It exposes a single tool, run, that takes the exact argv you'd pass to the shimguard CLI and returns a structured result ({returncode, stdout, stderr, json?} on success, {error: ...} if the command failed, timed out, or exited non-zero). Example call:

run(args=["verify", "sybil-solutions/codex-shim", "--issues", "45,46", "--format", "json"])

which returns the same --format json report shown above, as a parsed json field alongside the raw stdout. Full details are in the Python package README.

Library API

ShimGuard's verification logic is also importable directly:

import { TrackerVerifier, RestGitHubClient, RegexPatternMatcher } from "shimguard-cli";

const client = new RestGitHubClient(process.env.GITHUB_TOKEN);
const verifier = new TrackerVerifier(client, new RegexPatternMatcher(client));

const result = await verifier.verify({ owner: "sybil-solutions", repo: "codex-shim", number: 45 });
console.log(result.verdict); // "MISMATCH"

TrackerVerifier takes any GitHubClient and an optional PatternMatcher (see src/types.ts, src/pattern-matcher.ts), both are interfaces, so a future local-config scanner or a different code host can plug in without changing the verifier itself.

The Python package exposes the same shape:

from shimguard import TrackerVerifier, RestGitHubClient, RegexPatternMatcher, IssueRef

client = RestGitHubClient()  # or RestGitHubClient(token=os.environ["GITHUB_TOKEN"])
verifier = TrackerVerifier(client, RegexPatternMatcher(client))

result = verifier.verify(IssueRef(owner="sybil-solutions", repo="codex-shim", number=45))
print(result.verdict)  # "MISMATCH"

How it compares

No existing open-source tool checks "this issue tracker says fixed-in-PR-#N, is #N actually merged." That claim is verified by searching for issue-fix-verification tools, patch-verification tools, and security-advisory-fix tools before writing this. The closest adjacent tools solve different problems:

Read the full README on GitHub →