What litmus does
The litmus MCP server evaluates MCP servers through the way an agent would interact with them. It records the target's exact tool surface, runs behavioral checks, assigns an A–F grade, and writes a deterministic evidence bundle. Results include the findings supporting the grade rather than only a letter.
The harness accepts npm and PyPI references, github/owner/repo targets, HTTPS MCP endpoints, and local entry files. GitHub targets are cloned, built, and run in a sandbox, with the commit SHA pinned for the grade. A typical run takes about 20–60 seconds and exits with a non-zero status for D or F results.
The four probe categories cover tool-output injection, permission and egress behavior, sensitive-data handling, and adversarial inputs. The npm workflow can also inspect dependencies against the OSV vulnerability database. That dependency result is a separate, point-in-time advisory and does not change the behavioral grade.
How it works
The litmus MCP server first connects to the target and fingerprints its available tools. It then exercises the four probe groups: C-01 checks static, dynamic, and second-order tool-output injection; C-02 checks network permissions in a default-deny Docker environment; C-03 uses planted canaries to examine sensitive-data handling; and C-04 sends malformed, oversized, and jailbreak-style inputs.
Docker is required for the GitHub build-and-run path and for the egress isolation used by C-02. Without Docker, C-02 is skipped and the resulting grade is capped at B. Because grading executes the target's code, the README recommends treating target selection and CI discovery as security-sensitive operations.
Published grades can be read without executing the target. The hosted MCP endpoint provides lookup-only access, while local execution is needed for run_litmus and run_skill_litmus. Re-running the same command against the same target is intended to reproduce the result or provide evidence for disputing it.
Setup and configuration
Run a one-off grade with:
npx -y -p @polygraphso/litmus polygraphso-litmus litmus npm/@modelcontextprotocol/server-filesystem
The -p option selects the package because the npm distribution contains three command-line binaries. For CI or a pre-commit hook, use the package's ci command and pin a version, such as npx @polygraphso/litmus@0.20.0 ci.
For lookup-only use, configure an MCP client with https://polygraph.so/api/mcp. The package also includes the stdio server polygraphso-litmus-mcp, which exposes the full tool set to MCP-capable clients. Claude Code users can install the accompanying plugin and use its grade and check commands. Cursor can be configured through the documented manual JSON setup.
Tools and capabilities
The local MCP server exposes:
check_server to retrieve a published grade without running the target.list_servers to browse graded servers with pagination, grade filters, and corpus totals.request_grade to submit a paid grade request; the response includes the payment link.run_litmus to execute the behavioral harness and return the grade with evidence.run_skill_litmus to statically assess a Claude Code or Agent Skill as A, B, D, or F.verify_attestation to inspect the onchain proof for a published grade on Base.
The GitHub Action can fail builds when servers or skills fall below a configured minimum. It supports explicit targets, optional discovery, strict handling of ungradeable targets, a working directory, a version, and a bearer value. The same gate can run outside GitHub as a command.
Limitations and notes
A grade measures the tested behavior; it is not a guarantee of safety. Running a target still executes its code, even though egress is sandboxed. On public repositories, the README advises using explicit target allowlists, leaving discovery disabled, avoiding pull_request_target, and supplying bearer credentials only to trusted, pinned remotes.
Dependency auditing applies only to npm targets. It resolves the dependency tree with npm install --package-lock-only --ignore-scripts, does not download tarballs or run package code, and can be disabled with LITMUS_DEPS_AUDIT=0. Vulnerability data may change after a grade, so it is excluded from reproducible evidence.
The litmus MCP server's hosted endpoint serves lookup tools only. Local stdio installation is required when an agent needs to grade a target immediately or run the skill-specific check.