AI Guardian

Disclaimer: Community-maintained open-source project. Not affiliated with, endorsed by, or sponsored by Ollama, IGEL, or any AI-security vendor. Product and trademark names belong to their owners. MIT licensed.

Governed observability + governance for on-endpoint local LLMs. It lets you observe + audit what your local models are actually fed, and gate what leaves in a prompt — the complement to IGEL AI Armor. AI Armor governs whether a local model may run on the endpoint; ai-guardian records what it did and gates what goes into the prompt (secrets, PII, source, jailbreaks) plus which model may serve it. Self-contained: it talks to each runtime's REST API and needs nothing beyond httpx and the MCP SDK. v0.1 provides opt-in route-through content governance, plus a transparent capture proxy for clients that did not opt in.

Supported runtimes

One tool, several local runtimes, selected per target by a runtime field in config.yaml (the init wizard asks). Ollama uses its native API; the other three share one OpenAI-compatible transport (/v1/models + /v1/chat/completions).

The allow/deny model policy, the deterministic prompt scanner, the route-through guard (guarded_generate / observe_chat), provenance drift, and doctor work across all runtimes. Model lifecycle writes (pull / remove / unload) are Ollama-only — the OpenAI-compatible servers load a model at startup and expose no lifecycle endpoint, so those writes are refused with a clear message.

Provenance honesty: only Ollama (content digest) and llama.cpp (a /props-derived path/size identity) expose something to pin. LM Studio and vLLM expose only a model id, so a pinned digest is reported unverifiable rather than a false DRIFT.

vLLM here is a LOCAL endpoint-guarding use case. GPU inference-cluster operations (autoscale, drain, Ray Serve/Jobs, model lifecycle at fleet scale) belong to a different tool in the line — GPU cluster ops → inference-aiops.

What it does

Ollama persists no queryable prompt/response history — conversational context is client-supplied on every request. So ai-guardian observes on two fronts:

• Passive inventory / state auditing — over /api/tags, /api/ps, /api/show, /api/version: what models are installed and running, their VRAM residency, license/params/capabilities, and their provenance digests. Every model is annotated with an allow/deny policy verdict, so shadow (unsanctioned) models show allowed: false.
• Opt-in route-through content governance — callers send a prompt through ai-guardian (guarded_generate / observe_chat). It scans the text (secrets / PII / source / jailbreak), checks the model against policy, records the interaction to its own usage log (~/.ai-guardian/usage.db), and only then calls Ollama — blocking when the risk band is too high or the model is disallowed. The raw prompt is never stored (only its length + redacted findings).

Now available (ai-guardian proxy serve): a transparent capture proxy that applies the same scan + model policy + recording to traffic from clients that never opted in — point them at the proxy instead of the runtime. It inspects requests and streams responses through untouched, and it is a chokepoint, not an enforcement boundary: a client that can still reach the runtime's real port bypasses it entirely. proxy_guidance returns the command and that caveat; the CLI prints it on every start.

Key features

• Deterministic, offline prompt scanner — no I/O, no network, so it is fully testable offline. Flags secrets (AWS AKIA, private-key blocks, GitHub / Slack / OpenAI / Google tokens, JWTs, assigned api_key=…, high-entropy fallback), PII (email, US SSN, credit card with a Luhn check), source/config-leak heuristics, and jailbreak / prompt-injection signatures — rolled up into a weighted risk band (low / medium / high / critical; any critical dominates). Findings are redacted — the scanner never re-emits the secret it caught.
• Model allow/deny policy (shell-glob patterns) so shadow / unsanctioned models surface as allowed: false, plus provenance digest pinning to flag a model whose digest drifted (re-pulled / tampered).
• Route-through guard — guarded_generate / observe_chat scan + policy-gate
  • record + run-if-allowed, blocking on risk-band >= block_threshold (default high) or a disallowed model.
• Vendored governance harness — audit log, token/runaway budget guard, descriptive risk tiers, and undo-token recording, bundled in the package (no external dependency).
• Highly self-testable — Ollama is free + local for the API parts; the scanner, policy, and risk-band are pure deterministic offline logic.

What this tool does, and does not, decide

It delivers local-LLM observability and operations — reads and writes — accurately, and records every one of them. It does not decide whether a write to the model estate is allowed to happen. That is the agent's judgement, or the permission of the host and account you run it under: point it at a runtime the account cannot administer — an Ollama daemon whose model store the user can't modify, or an endpoint the agent reaches read-only — and the writes fail at the runtime, the place that actually owns the permission. Simplest of all, hand the connecting agent only the scan/observe tools.

So the harness has no read-only switch, no deny-rules file, and no approval gate to configure. (Content governance is a separate, product-level thing that stays: the model allow/deny policy and the guarded_generate block threshold still scan and gate what a model is asked to do.) The one thing the harness guarantees is that nothing is silent: every call, over MCP and over the CLI alike, lands an audit row in ~/.ai-guardian/audit.db, and destructive writes still capture their before-state and record an inverse where one exists.

Each tool declares a risk_level, kept in agreement with its [READ]/[WRITE] documentation tag by a test, and carried into the audit row as a descriptive tier — so a reviewer can see at a glance that a row was a high-risk delete. It is a label, not a gate.

Running a smaller / local model? See agent-guardrails.md — it lists the guardrails this tool now enforces for you (so you don't spend prompt budget restating them) and gives a ready-made system prompt for what's left.

Capability matrix (21 MCP tools)

Reads (11)

Writes (8)

Undo (2)

Risk-band gating: guarded_generate / observe_chat block when the prompt's risk band >= block_threshold (default high) or the model is disallowed. Blocked calls never reach Ollama and are recorded as blocked in the usage log.

Quick start

As a Claude Code plugin

One install gives an agent both the skill and the MCP server:

/plugin marketplace add AIops-tools/marketplace
/plugin install ai-guardian@aiops-tools

The MCP server is fetched with uv and pinned to the package version this plugin declares, so an audit row can be traced back to the code that wrote it. Credentials are still configured with ai-guardian init — see below.

As an OpenClaw plugin