Eval View logo

Eval View

by hidai25

Records AI-agent behavior, compares it with golden baselines, and flags tool-call or output regressions in CI.

Install Eval View

Pick your client, then add it in one click or paste the config.

STDIOPython
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "hidai25-eval-view": {
      "command": "uvx",
      "args": [
        "evalview"
      ]
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

Install not yet sandbox-tested— we haven't run this command through our automated sandbox yet. Not a red flag; we're working through the catalog.

Overview

hidai25/eval-view MCP server provides snapshot-style regression testing for AI agents by recording tool calls, parameters, ordering, and outputs. Its documented workflow uses the EvalView CLI to create a baseline with `snapshot` and compare later runs with `check`. Deterministic trajectory comparison runs offline, while optional judge and embedding-based checks can evaluate output quality.

Use cases

  • Record agent behavior before a release
  • Compare tool-call trajectories after prompt changes
  • Block regressions in pull requests
  • Test HTTP-based agents in CI
  • Monitor model or provider drift

Key features

  • Golden behavior snapshots
  • Full tool-call trajectory diffs
  • Offline deterministic comparison
  • Optional LLM judging and embeddings
  • CI regression gates
  • Multi-variant baselines

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked

Recent health check succeeded.

31/31 checks healthy over the last 102d

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 16d ago via OSV.dev · evalview (PyPI)

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars142Stargazers on the source repository.
Last commitTodayMost recent push to the default branch.
Directory activity3 viewsConfig copies, upvotes, and views on AllMCPs.

How Eval View works

What hidai25/eval-view MCP server does

The hidai25/eval-view MCP server project documents EvalView, a regression-testing framework for AI agents. It treats an agent run as a snapshot: the recorded behavior includes the tools used, their parameters, their order, and the resulting output. Later runs are compared with that stored baseline so changes can be reviewed as passed checks, tool changes, or regressions.

This approach is intended for cases where an agent can continue returning successful HTTP responses while its behavior has changed. For example, a model or prompt update may cause a different tool to be called, a clarification step to be skipped, or output quality to decline. A new baseline can be accepted with another snapshot run when the changed behavior is intentional.

How it works

The main workflow has two commands. evalview snapshot records the current behavior of the configured agent. evalview check executes the tests again and compares the results with the saved baseline. The comparison covers the complete tool-call trajectory rather than only the final response.

The core tool and sequence comparison is deterministic and does not require an API key. An LLM judge can be added when output-quality scoring is needed, and semantic comparison using embeddings is optional. Running the agent itself may still call a model or provider and therefore may incur provider charges; disabling the judge does not disable those agent calls.

EvalView supports multi-variant baselines, with up to five valid paths for nondeterministic behavior. It can also run statistical or pass@k evaluations, record and replay cassettes, check for model drift, monitor production behavior with Slack alerts, and generate regression tests from incidents. These are documented as additional features beyond the basic snapshot and check loop.

Setup and configuration

Install the Python package with:

Terminal
pip install evalview

A demonstration is available through evalview demo. For a configured agent, create a baseline first, then run checks after changes:

bash
evalview snapshot
evalview check

The project can test agents exposed through an HTTP endpoint, such as evalview check --agent http://localhost:8000/invoke. It also provides a Python API; importing gate from evalview allows a test directory to be evaluated programmatically, with a Boolean pass result and per-test diffs available on the returned value.

Tools and capabilities

The documented EvalView capabilities include:

  • Recording golden behavior snapshots.
  • Comparing tool names, parameters, order, and outputs.
  • Reporting changed tool trajectories and lower-quality results.
  • Running checks as a CI merge gate.
  • Posting pull-request comments containing diffs, cost and latency changes, and a pass/fail result through the supplied GitHub Actions example.
  • Testing LangGraph, CrewAI, OpenAI, Claude, Mistral, Ollama, MCP-based agents, and arbitrary HTTP APIs.
  • Running multi-turn tests, statistical evaluations, cassettes, drift canaries, production monitoring, and incident-based test generation.

Limitations and notes

The hidai25/eval-view MCP server README does not list MCP tool names, input schemas, transport settings, or compatibility instructions for Claude Desktop, Cursor, Windsurf, or Cline. Its concrete usage documentation describes the evalview CLI, Python library, HTTP agent endpoint, and CI action instead. Do not assume that installing the Python package alone exposes a standalone MCP transport.

The latest published release described in the README is version 0.8.1. Its OpenAI Assistants adapter still uses the Assistants API, while a Responses API migration is described as unreleased source. Users of openai-assistants therefore need to consult the migration guidance before testing. Other listed adapters are described as unaffected.

Provider credentials may be needed by the agent or optional evaluator, even though the deterministic comparison itself can run without an API key. Live provider checks can also involve paid API usage, while the local core checks use no paid inference calls.

Written by AllMCPs with AI assistance from the project’s README and listing data. How we build listings

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about Eval View

It does not document a separate MCP server launch command.

Run `evalview snapshot` to save the current behavior, then run `evalview check` after changes. The check compares tool calls, parameters, ordering, and outputs with the baseline.

Yes.

It mentions Claude as a supported agent or provider but gives no Claude Desktop configuration.

Related MCP servers

Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewEval View AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/hidai25-eval-view?style=directory)](https://allmcps.com/mcp/hidai25-eval-view)
HTML Embed
<a href="https://allmcps.com/mcp/hidai25-eval-view"><img src="https://allmcps.com/api/badge/hidai25-eval-view?style=directory" alt="Eval View on AllMCPs" /></a>