Log inSubmit server
Codabench MCP logo

Codabench MCP

Drive Codabench benchmark discovery, data downloads, submissions, status checks, and leaderboard queries through MCP.

InstallRepositoryWebsite
Full README Compare alternatives Embed widget Claim this listing

Install Codabench MCP

Pick your client, then add it in one click or paste the config.

STDIOPython
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "codabench-mcp": {
      "command": "uvx",
      "args": [
        "codabench-mcp"
      ],
      "env": {
        "CODABENCH_API_TOKEN": "YOUR_VALUE_HERE",
        "CODABENCH_BASE_URL": "YOUR_VALUE_HERE",
        "CODABENCH_ALLOW_WRITE_RAW": "YOUR_VALUE_HERE",
        "CODABENCH_MAX_DOWNLOAD_BYTES": "YOUR_VALUE_HERE"
      }
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

Values are written into the snippet above so you can paste it as is. They never leave your browser.

We couldn’t confirm it starts correctly

We ran the install command but it didn't respond within our test window — this can mean a slow first-time install rather than a real problem.

uvx codabench-mcp

Process exited before answering initialize.

This is an experimental automated check and can have false negatives — missing environment variables, a slow cold install, etc. It doesn’t necessarily mean something’s wrong. Last checked 1d ago.

Overview

codabench-mcp MCP server connects AI agents to the Codabench REST API for participant workflows. It provides focused tools to find competitions, inspect phases and rules, download datasets, submit work, monitor submissions, and read results. Authentication uses a Codabench API token supplied through an environment variable. Reach for it when an agent needs to operate a Codabench benchmark account rather than only retrieve static competition information.

Use cases

  • Discover Codabench competitions and inspect their rules
  • Download benchmark datasets for an agent workflow
  • Submit files to a Codabench competition phase
  • Monitor submission processing and retrieve logs
  • Read competition leaderboards and participant results

Key features

  • Competition, phase, rule, and task discovery
  • Streaming dataset downloads with SHA-256 verification
  • Three-step Codabench submission handling
  • Submission polling with backoff and timeout
  • Leaderboard, profile, and submission queries
  • Generic REST requests with GET-only default

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked Today

Recent health check succeeded.

5/5 checks healthy over the last 40d

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 14d ago via OSV.dev · codabench-mcp (PyPI)

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

Last commit4mo agoMost recent push to the default branch.
Install checkInconclusiveDidn't respond in our test window — often a slow first install.
Directory activity2 viewsConfig copies, upvotes, and views on AllMCPs.

How Codabench MCP works

What codabench-mcp does

The codabench-mcp MCP server exposes the Codabench REST API through tools designed around the participant journey for machine-learning benchmarks. An agent can search for competitions, inspect a selected competition and its phases, read rules, enumerate tasks, and retrieve task details. It can also access account-specific information such as the current user's profile and previous submissions.

The server covers the operational steps after competition discovery. It can download a dataset, send a submission to a phase, wait for processing, inspect submission logs, and retrieve leaderboard information. This makes it suitable for agents that need to move from understanding a benchmark to running a complete submission workflow.

How it works

At startup, the server reads a Codabench API token from the environment and sends it as a DRF token in the authorization header. The default API host is https://www.codabench.org, with an option to replace it for testing or another supported deployment.

Dataset downloads are streamed and checked with SHA-256. Submission handling follows Codabench's three-step upload process rather than requiring the agent to implement those HTTP steps itself. For asynchronous jobs, poll_submission uses backoff and a timeout; a submission that remains in progress is returned as a non-error state.

The codabench_request tool provides a lower-level REST option. It permits GET requests by default and can be configured to permit non-GET methods. This is useful for API paths not represented by the curated tools, but it also puts more responsibility on the caller.

Setup and configuration

Install the released package with uvx codabench-mcp. To run the repository's current main branch, use uvx --from git+https://github.com/harshitAgr/codabench-mcp codabench-mcp. A development checkout can be started after uv sync with uv run codabench-mcp.

Create a Codabench API token through the site's Swagger UI or by posting credentials to the token-auth endpoint. Set CODABENCH_API_TOKEN in the MCP client's environment.

CODABENCH_BASE_URL overrides the default host. CODABENCH_ALLOW_WRITE_RAW=1 enables non-GET calls through the raw request tool. CODABENCH_MAX_DOWNLOAD_BYTES limits dataset downloads; its documented default is 5 GB.

Tools and capabilities

The codabench-mcp MCP server includes tools for:

  • Searching and retrieving competitions, phases, rules, tasks, and task details.
  • Streaming datasets with a SHA-256 check.
  • Listing submissions and retrieving an individual submission or its logs.
  • Uploading a submission to a phase through Codabench's required flow.
  • Polling asynchronous submission status with backoff and timeout handling.
  • Reading leaderboards and the authenticated user's profile.
  • Making generic REST requests, GET-only by default.

Limitations and notes

A valid Codabench API token is required to run the server. The raw REST escape hatch does not allow non-GET methods unless its environment setting is enabled. Dataset retrieval is bounded by the configured download-size cap. The documented development tests run without network access, so they do not by themselves verify a live Codabench account or endpoint.

Read the full README View source on GitHub

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about Codabench MCP

codabench-mcp is an MCP server that connects AI agents to the Codabench REST API. Its main tools search competitions, inspect rules and tasks, download datasets, submit files, poll submission status, read logs and leaderboards, and retrieve the authenticated user's profile.

Install it with `uvx codabench-mcp`. You can also run the latest `main` branch with `uvx --from git+https://github.com/harshitAgr/codabench-mcp codabench-mcp`, or clone the repository, run `uv sync`, and start it with `uv run codabench-mcp`.

Set `CODABENCH_API_TOKEN` to a token obtained from Codabench's Swagger UI or token-auth endpoint. Claude Desktop receives it in the MCP server configuration, while Claude Code can receive it through an exported shell variable or the `claude mcp add` command.

Claude Desktop uses an `mcpServers` entry that launches `uvx codabench-mcp`; Claude Code supports a plugin marketplace installation or manual MCP registration.

Related MCP servers

All in Data Science Tools All alternatives
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewCodabench MCP AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/codabench-mcp?style=directory)](https://allmcps.com/mcp/codabench-mcp)
HTML Embed
<a href="https://allmcps.com/mcp/codabench-mcp"><img src="https://allmcps.com/api/badge/codabench-mcp?style=directory" alt="Codabench MCP on AllMCPs" /></a>