Log inSubmit server
MCP Server logo

MCP Server

by iris-eval

Scores agent traces locally for quality, safety, cost, PII, and citation issues with deterministic rules and optional LLM judging.

InstallRepositoryWebsite
Full README Compare alternatives Embed widget Claim this listing

Install MCP Server

Pick your client, then add it in one click or paste the config.

STDIONode.js
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "iris-eval-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ]
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

Automated check passed— started and listed 12 tools correctly (1d ago).

Overview

iris-eval/mcp-server MCP server records agent traces in a local SQLite database and evaluates outputs against inspectable rules for quality, safety, and cost. Its MCP tools support trace logging, retrieval, rule management, run comparison, citation checks, deterministic scoring, and optional LLM-judge evaluations. A local dashboard shows failures, evidence, scores, and run comparisons, while HTTP ingestion can capture traces without relying on a model to call an MCP tool. Reach for it when you need auditable agent evaluation on a developer machine without an account or SDK.

Use cases

  • Log agent runs for later inspection
  • Evaluate outputs against deterministic safety and quality rules
  • Compare runs and traces across agent versions
  • Review failures and evidence in a local dashboard
  • Apply optional LLM judging to semantic evaluation tasks

Key features

  • Local SQLite trace storage
  • Deterministic output and tool-call evaluation
  • Configurable evaluation rule management
  • Optional Anthropic or OpenAI LLM judging
  • Run and trace comparison
  • Dashboard and HTTP trace ingestion

Tools12

What MCP Server lets your agent do. Expand a tool for its parameters.

Verified live~8.9k tokens of context

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked 2d ago

Recent health check succeeded.

19/19 checks healthy over the last 78d

What it can do

From the 12 tools this server returned when we ran it; 12 declare their own safety hints.

Can delete or modify data · 4Handles credentials · 1Makes network requests · 4Read-only tools · 8
Network at startup3 connections
Files written at startup3
Install scriptsNone
npm provenanceSigned build
Package age226 days
Maintainers1

Observed 2026-10-26 in an isolated sandbox. A description of reach, not a security audit.

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 7d ago via OSV.dev · @iris-eval/mcp-server (npm)

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars9Stargazers on the source repository.
npm downloads2.9kPackage downloads in the last 30 days.
Last commit5d agoMost recent push to the default branch.
Availability100%Our rolling endpoint + install checks that succeeded.
Install checkPassedOur sandbox started it and listed its tools.
Tools exposed12Callable tools this server registers over MCP.
Directory activity3 viewsConfig copies, upvotes, and views on AllMCPs.

How MCP Server works

What iris-eval/mcp-server MCP server does

iris-eval/mcp-server MCP server provides local evaluation and observability for agent runs. It stores traces in SQLite on the machine running the server, then applies deterministic checks to agent outputs, tool calls, costs, PII, prompt-injection indicators, hallucination markers, and other evaluation conditions. Rules are inspectable and can be deployed, paused, edited, or removed.

The server also supports an optional LLM judge for semantic evaluations. Anthropic or OpenAI credentials can be supplied for that judge, which applies a per-evaluation cost limit. Citation verification can fetch pages referenced by an output, allowing cited claims to be checked against their sources.

How it works

Agents can call MCP tools such as log_trace, evaluate_output, and get_traces. Additional tools expose rule listing and deployment, trace deletion, run and trace comparison, scoring, citation verification, and LLM-judge evaluation. The resulting verdicts and rule evidence are available through the local dashboard.

MCP calls are not an automatic interception layer: a model or application must request trace logging. For capture that should not depend on model behavior, an application can send traces to POST /api/v1/traces over HTTP. The endpoint can also run deterministic evaluations in the same request. It is served on the dashboard port, which defaults to port 6920, and is available only while the dashboard is running.

The dashboard presents failures and their evidence, supports viewing individual failures, and provides comparisons for runs over the same questions. All configured MCP clients use the shared Iris database, so traces can be viewed together across installations.

Setup and configuration

Node.js 22.13 or newer is required. A quick health check is available with:

Terminal
npx @iris-eval/mcp-server --self-test

Run the demo with npx @iris-eval/mcp-server --demo; it seeds a separate demo database and serves the dashboard at http://localhost:6920. Remove that data with --demo-clear.

For an MCP client, the installer can write the client’s configuration while preserving other configured servers:

Terminal
npx -y @iris-eval/mcp-server install claude-code

Use install --list to see detected clients, or install <client> --uninstall to remove the configuration. A manual configuration can start the package with npx -y @iris-eval/mcp-server; adding --dashboard also starts the dashboard. The dashboard can be enabled with IRIS_DASHBOARD=true.

The LLM judge accepts an Anthropic or OpenAI API key through the corresponding Iris environment variable. Without those keys, deterministic evaluations remain available.

Limitations and notes

Iris runs locally and does not send data elsewhere by default. Data leaves the machine when an OpenTelemetry endpoint, an LLM judge, citation checking, or a webhook is explicitly enabled. The LLM judge sends judged text to the selected provider; citation checks fetch cited pages. Webhook notifications contain identifiers, verdicts, and rule names rather than trace text.

The dashboard’s HTTP ingest endpoint is not available merely because the MCP transport uses HTTP; the dashboard must be started separately. The server’s MCP tools also do not automatically observe an agent’s activity, so applications requiring guaranteed capture should use the HTTP ingest endpoint or call the logging tools directly.

Read the full README View source on GitHub

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about MCP Server

iris-eval/mcp-server is an MCP server that connects agent workflows to a local SQLite evaluation store and dashboard. Its main tools log and retrieve traces, evaluate outputs, manage rules, compare runs, verify citations, calculate scores, and optionally use Anthropic or OpenAI as an LLM judge.

Install it in a supported MCP client with `npx -y @iris-eval/mcp-server install <client>`. The installer writes the client configuration while retaining other servers; `install --list` shows detected clients, and `install <client> --uninstall` removes Iris.

Yes, iris-eval/mcp-server works with Cursor. Cursor is one of the clients supported by the installer, and the repository also provides a Cursor installation link. After configuration, restart the client so it loads the server.

Send the trace to `POST /api/v1/traces` over HTTP while the dashboard is running. The endpoint listens on the dashboard port, which defaults to 6920, and can run deterministic evaluations in the same request.

Provide an Anthropic or OpenAI API key through the corresponding Iris environment variable, then use the LLM-judge evaluation tool. The judge is optional, and deterministic rules work without an LLM key.

Related MCP servers

All in Monitoring All alternatives
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewMCP Server AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/iris-eval-mcp-server?style=directory)](https://allmcps.com/mcp/iris-eval-mcp-server)
HTML Embed
<a href="https://allmcps.com/mcp/iris-eval-mcp-server"><img src="https://allmcps.com/api/badge/iris-eval-mcp-server?style=directory" alt="MCP Server on AllMCPs" /></a>