signet-eval

Deterministic policy enforcement for AI agent tool calls. Every action an agent proposes passes through user-defined rules before execution. No LLM in the authorization path. Advisory nudges are separate from authorization. 25ms end-to-end.

Install

# crates.io
cargo install signet-eval

# from source
git clone https://github.com/jmcentire/signet-eval
cd signet-eval
cargo install --path .

There is no npm or PyPI package for signet-eval. The public distribution path is crates.io plus source install from GitHub. The MCP Registry listing points at the repository metadata; the runtime is the local signet-eval serve stdio server.

Quick Start

1. Hook into Claude Code — add to ~/.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "",
      "hooks": [{"type": "command", "command": "signet-eval", "timeout": 2000}]
    }]
  }
}

For Codex, enable hooks in ~/.codex/config.toml or <repo>/.codex/config.toml:

[features]
codex_hooks = true

Then add ~/.codex/hooks.json or <repo>/.codex/hooks.json:

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "signet-eval --adapter codex",
        "timeout": 30000,
        "statusMessage": "Checking Signet policy"
      }]
    }],
    "PermissionRequest": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "signet-eval --adapter codex-permission",
        "timeout": 30000,
        "statusMessage": "Checking Signet approval policy"
      }]
    }]
  }
}

For Antigravity, add this block to ~/.gemini/config/hooks.json (merge it with any existing hook groups):

{
  "signet": {
    "enabled": true,
    "PreToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "/bin/bash -lc 'source ~/.profile >/dev/null 2>&1 || true; exec signet-eval --adapter antigravity'",
        "timeout": 30,
        "statusMessage": "Checking Signet policy"
      }]
    }]
  }
}

The same configuration is available at hooks/antigravity-hooks.json.

2. Done. Every tool call now passes through policy evaluation. The default policy blocks destructive operations, protects its own configuration, and allows everything else.

3. (Optional) Customize — talk to Claude with the MCP server:

claude mcp add --scope user --transport stdio signet -- signet-eval serve

Then say: "Add a $50 limit for amazon orders" or "Block all rm commands".

Default Policy

Self-protection rules are locked — they cannot be removed, edited, or reordered by the AI agent, even through the MCP management server. This prevents the agent from disabling its own guardrails.

Custom Policy

signet-eval init       # write default policy to ~/.signet/policy.yaml
signet-eval validate   # check policy for errors
signet-eval rules      # show current rules

Edit ~/.signet/policy.yaml:

version: 1
default_action: ALLOW
rules:
  - name: block_rm
    tool_pattern: ".*"
    conditions: ["contains(parameters, 'rm ')"]
    action: DENY
    reason: "File deletion blocked"

  - name: books_limit
    tool_pattern: ".*purchase.*"
    conditions:
      - "param_eq(category, 'books')"
      - "spend_plus_amount_gt('books', amount, 200)"
    action: DENY
    reason: "Books spending limit ($200) exceeded"

  - name: protect_my_config
    tool_pattern: ".*"
    conditions: ["contains(parameters, '/etc/')"]
    action: ASK
    locked: true
    reason: "System config changes require confirmation"

Rules are evaluated in order — first match wins. Multiple conditions on a rule are AND'd. Rules with locked: true cannot be modified through the MCP management server.

Advisory Injection

INJECT rules probabilistically add advisory context near the tool call that triggered them. They are nudges, not authorization: the normal ALLOW/DENY/ASK/GATE/ENSURE pass remains first-match-wins and deterministic. Injection runs afterward and only emits context when a matching inject rule fires.

rules:
  - name: maybe_remind_kindex_on_git
    tool_pattern: "^Bash$"
    conditions: ["contains(parameters, 'git ')"]
    action: INJECT
    inject:
      trigger:
        mode: exponential
        peak: 0.35
        cooldown_seconds: 300
        peak_after_seconds: 1800
        max_per_session: 3
      payload:
        text: "Before committing, check whether project `.kin` files should be included."

Trigger modes:

Payload sources:

Template substitutions are enabled by default: {tool_name}, {cwd}, {date}, and {matched_param.X}. See examples/inject_examples.yaml.

Condition Functions

Encrypted Vault

Three-tier encrypted storage with passphrase-derived key hierarchy (Argon2id + AES-256-GCM):

signet-eval setup                      # create vault with passphrase
signet-eval store cc_visa 4111...      # store Tier 3 credential
signet-eval status                     # vault status and spending
signet-eval log                        # recent action log
signet-eval unlock                     # refresh session after timeout

Credentials support scoped access via request_capability: domain restrictions, purpose constraints, per-use amount caps, and one-time tokens that auto-invalidate after a single use.

Spending limits use the vault ledger — each tool call that spends money is logged, and spend_plus_amount_gt() checks cumulative totals before allowing the next purchase.

Self-Protection

signet-eval ships with locked rules that prevent an AI agent from disabling its own policy enforcement:

1. protect_signet_dir — Denies any Write, Edit, or Bash command touching .signet/ (policy files, vault, HMAC)

2. protect_signet_binary — Denies tampering with the signet-eval binary itself

3. protect_hook_config — Requires user confirmation before modifying settings.json (where the hook is configured)

4. protect_signet_process — Denies kill/pkill/killall commands targeting signet processes

5. protect_preflight_storage — Denies agent-side mutation of preflight records

6. require_kindex_engagement_before_edits — Denies direct edit tools until durable session context is recorded

7. prefer_persistent_task_store — Denies ephemeral Task* state and points agents to Kindex tasks

8. protect_checks_dir — Denies agent-side replacement of trusted ENSURE scripts

9. protect_vault_passphrase — Reserves vault setup and unlock operations for the human

10. protect_signet_symlink — Denies symlink bypasses targeting protected enforcement surfaces