methods-mcp

Lightweight, on-demand MCP server for structured methods extraction + reproducibility heuristics on academic papers. Built for the Worldwide AI Science Fellowship build challenge.

⚠️ Status: alpha (0.1.x). The tool surface and output shapes may shift between minor versions. Pin to an exact version in production. Bug reports very welcome via GitHub Issues.

Quick demo

$ uvx --from methods-mcp methods-mcp --version
methods-mcp 0.1.6

# In a Claude Code session:
> /mcp add methods-mcp methods-mcp
> Run methods_repro_review on https://arxiv.org/abs/2509.06917

  → tool: methods_repro_review({"input_str":"https://arxiv.org/abs/2509.06917"})

# Returns a MethodsReproReview object. Read `narrative` first — it explains
# everything else in plain English, so no tool-learning is required:

{
  "status": "ok",
  "narrative":
    "Resolved the paper: 'Paper2Agent' by Miao et al. (arxiv 2509.06917, "
    "2025-09-08). Extracted 11 methods steps at moderate self-reported "
    "confidence (0.72) — the procedure is clearly described but hyperparameters "
    "and software versions are absent. Detected the associated code repository "
    "https://github.com/jmiao24/Paper2Agent from an inline link in the paper "
    "text (detection confidence 0.94). The repo scored 0.90/1.00 on the "
    "reproducibility heuristic — verdict: likely reproducible. Present signals: "
    "substantive README, dependencies file, notebooks, figure-plotting script, "
    "recent activity, permissive license. Missing: data/fixtures directory. "
    "Suggested entrypoint: `python make_figures.py`.",
  "metadata":          { ... },   # PaperMetadata
  "methods":           { ... },   # MethodsStructured (null if extraction failed)
  "code_repo":         { ... },   # CodeRepo           (null only if input unresolvable)
  "repro_assessment":  { ... },   # ReproAssessment   (null if no repo detected)
  "errors":            []         # [{step, error_type, message, hint}] on partial
}

methods-mcp is a small, sharply-scoped Model Context Protocol server. It gives any AI agent (Claude Code, Claude Desktop, your Agent SDK script, etc.) eight tools that turn an academic paper URL into:

• canonical metadata,
• best-effort full text + section split,
• a Pydantic-validated structured methods object (steps / reagents / equipment / analyses),
• the paper's associated code repository (best-effort discovery),
• a no-execution-required reproducibility verdict for that repo, and
• a multi-mode summary.

The wedge: heavyweight pipelines like Paper2Agent (Stanford) take 30 minutes to hours to digest a paper into agent-ready tools. methods-mcp is the agent-callable, on-demand complement — every tool returns in seconds, no clone, no execution.

Install

uv add methods-mcp
# or, install globally:
uv tool install methods-mcp
# or, classic pip:
pip install methods-mcp

API keys

For best performance, set both:

export ANTHROPIC_API_KEY=sk-ant-...
export GITHUB_TOKEN=ghp_...          # optional but recommended

Neither key is logged or persisted — they're sent only to api.anthropic.com and api.github.com respectively. See SECURITY.md.

Use it from Claude Code

/mcp add methods-mcp methods-mcp

Then in any Claude Code chat:

Take https://arxiv.org/abs/2509.06917 and run methods_repro_review. Summarise what the paper does, the methods steps, and how reproducible the repo looks.

Use it from the Claude Agent SDK

from claude_agent_sdk import ClaudeAgentOptions, ClaudeSDKClient

options = ClaudeAgentOptions(
    mcp_servers={
        "methods-mcp": {
            "type": "stdio",
            "command": "methods-mcp",
            "args": [],
        }
    },
    allowed_tools=["mcp__methods-mcp__methods_repro_review"],
)

async with ClaudeSDKClient(options=options) as client:
    await client.query(
        "Run methods_repro_review on https://arxiv.org/abs/2509.06917 "
        "and tell me whether the repo looks reproducible."
    )
    async for msg in client.receive_response():
        print(msg)

Tools

All tools return Pydantic v2 models (validated, JSON-serialisable). See src/methods_mcp/schemas.py for the full type surface.

Design notes

• extract_methods uses Anthropic tool-use to coerce the model into emitting an instance of the MethodsStructured Pydantic schema. On validation failure we send one repair message with the validation error and try again before raising.
• assess_repo_reproducibility does not clone or execute anything. It scores the repo from publicly-readable GitHub metadata + the recursive tree listing. This is the deliberate wedge against batch tools that try to actually rerun the paper.
• fetch_paper_text prefers ar5iv HTML over PDF parsing for arXiv papers. Falls back to pypdf for non-arXiv inputs.
• The default model is claude-sonnet-4-6. Override via METHODS_MCP_MODEL env var or per-call model= arg.
• methods_repro_review returns a self-describing response. Every call sets a top-level status ("ok" / "partial" / "empty") and a narrative string that summarises everything retrieved in plain English — including every numeric score in context. A reader who reads only narrative + status gets the full picture without needing to learn the sub-object shapes. Sub-objects can be null when unavailable (e.g. repro_assessment: null on a paper with no detected repo — status stays "ok" because "no repo" isn't a failure). Failed sub-steps contribute a structured entry to errors with {step, error_type, message, hint}, where hint is an actionable plain-English suggestion for recognised patterns (missing API keys, rate-limits, 404s, timeouts, etc.) and null otherwise.

Scores & verdicts explained

Tool outputs contain three numeric fields that look similar but mean very different things. They are triage signals for an agent deciding whether a paper is worth digging into, not calibrated claims about correctness.