The full upstream README, mirrored here for reference. Install config, tool schemas, adoption signals, and an original overview live on the AI Guardian listing page.
Disclaimer: Community-maintained open-source project. Not affiliated with, endorsed by, or sponsored by Ollama, IGEL, or any AI-security vendor. Product and trademark names belong to their owners. MIT licensed.
Governed observability + governance for on-endpoint local LLMs. It lets you
observe + audit what your local models are actually fed, and gate what leaves in
a prompt — the complement to IGEL AI Armor. AI Armor governs whether a
local model may run on the endpoint; ai-guardian records what it did and gates
what goes into the prompt (secrets, PII, source, jailbreaks) plus which model
may serve it. Self-contained: it talks to each runtime's REST API and needs
nothing beyond httpx and the MCP SDK. v0.1 provides opt-in route-through
content governance, plus a transparent capture proxy for clients that did not opt in.
One tool, several local runtimes, selected per target by a runtime field in
config.yaml (the init wizard asks). Ollama uses its native API; the other three
share one OpenAI-compatible transport (/v1/models + /v1/chat/completions).
| Runtime | runtime | Default port | List / policy | Scan + route-through guard | Provenance |
|---|---|---|---|---|---|
| Ollama | ollama | 11434 | ✅ | ✅ | digest (content hash — strong) |
llama.cpp (llama-server) | llamacpp | 8080 | ✅ | ✅ | props — /props model path/size → pinnable id |
| LM Studio | lmstudio | 1234 | ✅ | ✅ | id only — weaker; pins report unverifiable |
| vLLM (local single-node) | vllm | 8000 | ✅ | ✅ | id only — weaker; pins report unverifiable |
The allow/deny model policy, the deterministic prompt scanner, the route-through
guard (guarded_generate / observe_chat), provenance drift, and doctor work
across all runtimes. Model lifecycle writes (pull / remove / unload)
are Ollama-only — the OpenAI-compatible servers load a model at startup and expose
no lifecycle endpoint, so those writes are refused with a clear message.
Provenance honesty: only Ollama (content digest) and llama.cpp (a /props-derived
path/size identity) expose something to pin. LM Studio and vLLM expose only a model
id, so a pinned digest is reported unverifiable rather than a false DRIFT.
vLLM here is a LOCAL endpoint-guarding use case. GPU inference-cluster operations (autoscale, drain, Ray Serve/Jobs, model lifecycle at fleet scale) belong to a different tool in the line — GPU cluster ops → inference-aiops.
Ollama persists no queryable prompt/response history — conversational context is client-supplied on every request. So ai-guardian observes on two fronts:
/api/tags, /api/ps,
/api/show, /api/version: what models are installed and running, their VRAM
residency, license/params/capabilities, and their provenance digests. Every
model is annotated with an allow/deny policy verdict, so shadow
(unsanctioned) models show allowed: false.guarded_generate / observe_chat). It scans the text
(secrets / PII / source / jailbreak), checks the model against policy,
records the interaction to its own usage log (~/.ai-guardian/usage.db),
and only then calls Ollama — blocking when the risk band is too high or the
model is disallowed. The raw prompt is never stored (only its length + redacted
findings).Now available (
ai-guardian proxy serve): a transparent capture proxy that applies the same scan + model policy + recording to traffic from clients that never opted in — point them at the proxy instead of the runtime. It inspects requests and streams responses through untouched, and it is a chokepoint, not an enforcement boundary: a client that can still reach the runtime's real port bypasses it entirely.proxy_guidancereturns the command and that caveat; the CLI prints it on every start.
AKIA, private-key blocks,
GitHub / Slack / OpenAI / Google tokens, JWTs, assigned api_key=…, high-entropy
fallback), PII (email, US SSN, credit card with a Luhn check),
source/config-leak heuristics, and jailbreak / prompt-injection
signatures — rolled up into a weighted risk band (low / medium / high /
critical; any critical dominates). Findings are redacted — the scanner never
re-emits the secret it caught.allowed: false, plus provenance digest pinning to flag a
model whose digest drifted (re-pulled / tampered).guarded_generate / observe_chat scan + policy-gate
block_threshold (default
high) or a disallowed model.It delivers local-LLM observability and operations — reads and writes — accurately, and records every one of them. It does not decide whether a write to the model estate is allowed to happen. That is the agent's judgement, or the permission of the host and account you run it under: point it at a runtime the account cannot administer — an Ollama daemon whose model store the user can't modify, or an endpoint the agent reaches read-only — and the writes fail at the runtime, the place that actually owns the permission. Simplest of all, hand the connecting agent only the scan/observe tools.
So the harness has no read-only switch, no deny-rules file, and no approval gate
to configure. (Content governance is a separate, product-level thing that stays:
the model allow/deny policy and the guarded_generate block threshold still
scan and gate what a model is asked to do.) The one thing the harness guarantees
is that nothing is silent: every call, over MCP and over the CLI alike, lands
an audit row in ~/.ai-guardian/audit.db, and destructive writes still capture
their before-state and record an inverse where one exists.
Each tool declares a
risk_level, kept in agreement with its[READ]/[WRITE]documentation tag by a test, and carried into the audit row as a descriptive tier — so a reviewer can see at a glance that a row was a high-risk delete. It is a label, not a gate.
Running a smaller / local model? See agent-guardrails.md — it lists the guardrails this tool now enforces for you (so you don't spend prompt budget restating them) and gives a ready-made system prompt for what's left.
| Tool | Risk | What it returns |
|---|---|---|
list_models | low | installed models, each with the allow/deny verdict (shadow → allowed:false) |
running_models | low | loaded models: VRAM footprint + residency expiry |
model_details | low | license / parameters / capabilities for one model |
server_status | low | Ollama reachability + version |
vram_usage | low | total VRAM used by loaded models; flag over-budget |
policy_view | low | current allow/deny policy + provenance digest pins |
model_provenance | low | each installed digest vs its pin; flag drift |
scan_prompt | low | pure text scan → findings + weighted risk band (no model call) |
usage_events | low | query the observed-usage log |
anomaly_report | low | rollup: shadow models, digest drift, high-risk + blocked prompts |
proxy_guidance | low | the proxy serve command + the client change, and the explicit caveat that the proxy is a chokepoint, not an enforcement boundary; writes nothing, starts nothing |
| Tool | Risk | Undo / safety |
|---|---|---|
pull_model | medium | refused if it violates policy |
remove_model | high | dry-run + undo (re-pull) |
unload_model | medium | evict from VRAM (keep_alive:0) |
set_model_allowlist | medium | undo → prior allowlist |
set_model_denylist | medium | undo → prior denylist |
pin_model_digest | medium | pin a model's expected provenance digest |
guarded_generate | medium | the route-through guard: scan + policy-gate + record + run-if-allowed |
observe_chat | medium | same, for /api/chat messages |
| Tool | Risk | What it does |
|---|---|---|
undo_list | low | list recorded undo tokens |
undo_apply | medium | replay a recorded inverse descriptor |
Risk-band gating: guarded_generate / observe_chat block when the prompt's
risk band >= block_threshold (default high) or the model is disallowed.
Blocked calls never reach Ollama and are recorded as blocked in the usage log.
One install gives an agent both the skill and the MCP server:
The MCP server is fetched with uv and pinned to the
package version this plugin declares, so an audit row can be traced back to the
code that wrote it. Credentials are still configured with ai-guardian init — see below.
The same bundle is published on ClawHub, where one install delivers the skill and its MCP server together:
Restart the OpenClaw gateway afterwards so it loads the plugin. The MCP server is
fetched with uv, pinned to this exact release, so
uvx has to be on PATH — without it the skill still installs but reports
Visible to model: no. Credentials are configured exactly as below.
The command and the package have different names. The CLI, the skill and this repo are all
ai-guardian; the PyPI distribution isai-guardian-aiopsbecauseai-guardianwas already taken. Both spellings below are correct and intentional — installai-guardian-aiops, then runai-guardian.
Route a prompt through the guard (scan + policy-gate + record + run-if-allowed) via MCP:
Run as an MCP server (stdio) — the full 21-tool surface; the CLI is a convenience subset:
Where that password then lives: an exported variable is readable by every process this shell starts and is recorded by shell history. On a shared or long-lived host, prefer the interactive prompt, or inject it from a secret manager for the life of the one command that needs it.
Every operation — MCP and CLI — passes through the bundled @governed_tool
harness. It records; it does not authorize (see above).
~/.ai-guardian/audit.db
(relocatable via AI_GUARDIAN_AIOPS_HOME). This is separate from
~/.ai-guardian/usage.db, which holds the observed local-LLM usage.AI_GUARDIAN_RUNAWAY_MAX=0; optional
hard ceilings via AI_GUARDIAN_MAX_TOOL_CALLS / AI_GUARDIAN_MAX_TOOL_SECONDS.risk_level; it gates nothing.ai-guardian doctor is the
fastest live check; see docs/VERIFICATION.md for
exactly which boxes are ticked.Want a passive capture proxy, another scanner signature, a richer policy model, or an AI Armor hook? Open an issue or PR — feedback and contributions welcome.