Evalscopebench MCP

What evalscopebench-mcp MCP server does

The evalscopebench-mcp MCP server provides a remote interface to an AI SDK benchmark dashboard. It is designed for AI agents and coding teams that need structured benchmark operations rather than direct browser interaction. The hosted service exposes four MCP tools: run_benchmark_gate, compare_model_scores, read_benchmark_report, and issue_benchmark_receipt.

The service also provides a public server card and usage receipts. Its documentation describes the resulting workflow evidence as suitable for audit-oriented use, but the repository does not include the private production implementation.

How it works

Requests are sent to the hosted MCP endpoint at https://evalscopebench.clauxel.com/mcp. The endpoint uses Streamable HTTP, so an MCP client must support remote Streamable HTTP servers. Calls use JSON-RPC and production requests must include a bearer token in the Authorization header.

The server returns structured JSON from its tools. The available operations cover benchmark gate execution, score comparison, report reading, and receipt issuance. The public server card is available at /.well-known/mcp/server-card.json, and the service uses the registry name com.clauxel.evalscopebench/evalscopebench-mcp.

Setup and configuration

To use the evalscopebench-mcp MCP server:

1. Open the service website and select a plan.
2. Create or request an API token.
3. Configure an MCP client that supports Streamable HTTP remote servers with the MCP endpoint.
4. Send requests with the token as a bearer credential.

The required HTTP header is:

Authorization: Bearer <token>

The provided material does not specify environment variable names or a local installation process. The repository serves as public documentation and a directory reference; it does not contain the hosted service's private production source code.

Tools and capabilities

• run_benchmark_gate — exposes the benchmark gate operation.
• compare_model_scores — exposes model score comparison.
• read_benchmark_report — exposes benchmark report reading.
• issue_benchmark_receipt — exposes benchmark receipt issuance.

Together, these tools cover the documented benchmark workflow. The available material does not define each tool's parameters, scoring rules, report schema, or receipt fields, so those details should be confirmed through the service's current documentation or server metadata before implementation.

Limitations and notes

The evalscopebench-mcp MCP server is a paid hosted service, not a self-contained server that can be run from this repository. Production calls require a token issued through the product website. An unauthenticated browser request to the MCP path returns a JSON error rather than internal service details.

The repository does not document offline operation, local deployment, specific model providers, or benchmark datasets. It also does not name particular MCP client applications; compatibility depends on whether the chosen client supports remote Streamable HTTP servers and bearer-authenticated MCP requests.