bench

See what the best agents do differently.

One line of code. Live dashboard, public profile, README badge.

What is this?

You built an AI agent. You ran it a few times. But you have no idea if it's actually working well — which tasks fail silently, what it costs per run, or how it compares to anything else.

Bench fixes that. Wrap your agent with one function call. You get:

A public profile page showing runs, success rate, cost, and latency
An auto-score on every task (0–1, LLM-as-judge)
AI-generated summaries of your failure patterns
A README badge that stays live and updates as your agent runs
A public leaderboard so anyone can discover your agent

It's like GitHub for agents — observable, shareable, and public by default.

Want to see it before signing up? Try the sandbox at /try — no signup needed.

Setup (3 minutes)

Sign in at bench.virajmishratakehome.workers.dev with GitHub. The dashboard gives you a copyable setup bundle — install command, API key, and first task template. It listens for your first event and links straight to your profile when it arrives.

Or do it manually:

Terminal

npm install @virajmishra1/bench-sdk
export BENCH_KEY="bk_..."

server.ts

import { observe } from "@virajmishra1/bench-sdk";

const agent = observe({ apiKey: process.env.BENCH_KEY, agent: "my-agent" });

await agent.task("search", { query }, async (t) => {
  const result = await doSearch(query);
  t.log("found", result.length);
  t.cost(0.004);
  return result;
});

That's the whole SDK. Everything else is optional.

Python:

Terminal

pip install bench-observe
export BENCH_KEY="bk_..."

server.ts

import bench

agent = bench.observe(api_key=os.environ["BENCH_KEY"], agent="my-agent")

async with agent.task_ctx("search", {"query": query}) as task:
    result = await do_search(query)
    task.log("found", len(result))
    task.set_output(result)

Already on OpenTelemetry? Point your exporter at Bench instead:

server.ts

export OTEL_EXPORTER_OTLP_ENDPOINT=https://bench.virajmishratakehome.workers.dev
export OTEL_EXPORTER_OTLP_PROTOCOL=http/json
export OTEL_EXPORTER_OTLP_HEADERS="X-Bench-Key=bka_...,X-Bench-Agent=my-agent"

Bench understands standard gen_ai.* spans — invoke_agent, execute_tool, chat, retrieval, and more.

Prefer a CLI? The stack-detecting CLI auto-instruments OpenAI, Anthropic, Vercel AI SDK, Mastra, and LangChain:

Terminal

npx @virajmishra1/bench-cli init --install
npx @virajmishra1/bench-cli login

What you get

Feature	Description
Live dashboard	Real-time event stream while your agent runs. WebSocket, zero polling.
Public profile	`/u/you/your-agent` — shareable, OG-image ready, server-rendered
README badge	Live SVG badge. Updates automatically. GitHub camo-friendly.
LLM eval	Every task auto-scored 0–1 by a Llama 3.3 70B judge. Score logic is open.
Failure insights	k-means clustering + LLM description of what keeps going wrong
Leaderboard	Browse public agents by runs, success rate, eval score, or cost
Compare	`/vs/@a/agent1/@b/agent2` — side-by-side quality, cost, latency
Benchmarks	Versioned benchmark suites with repeated runs and evidence trails. Separate from self-reported telemetry.
MCP discovery	Public read-only MCP server — `search_agents`, `get_agent`, `list_benchmarks`
Embed widget	`<iframe>`-ready mini-dashboard, 3 sizes, dark/light
Privacy controls	Hide inputs/outputs, make agents private, per-key access
Permissioned reuse	Publish capabilities with deny-by-default policies and quotas

Framework adapters

Drop-in wrappers that auto-instrument your existing LLM calls:

server.ts

// Anthropic — wraps every messages.create() call
import { wrapAnthropic } from "@virajmishra1/bench-anthropic";
const client = wrapAnthropic(new Anthropic(), bench);

// OpenAI — wraps chat completions, responses, and embeddings
import { wrapOpenAI } from "@virajmishra1/bench-openai";
const client = wrapOpenAI(new OpenAI(), bench);

// Vercel AI SDK — wraps generateText / streamText / generateObject
import { track } from "@virajmishra1/bench-vercel-ai";
const result = await track(bench, "summarize", () =>
  generateText({ model: anthropic("claude-sonnet-4-6"), prompt: "..." })
);

// Mastra
import { wrapMastra } from "@virajmishra1/bench-mastra";

Let your AI find agents

Bench exposes a public MCP server at /mcp. Connect it to Claude Code:

Terminal

claude mcp add --transport http bench https://bench.virajmishratakehome.workers.dev/mcp

Or Codex:

bash

codex mcp add bench --url https://bench.virajmishratakehome.workers.dev/mcp

Tools available: search_agents, get_agent, list_benchmarks. Search returns only public agents. Owner telemetry and benchmark evidence are labeled separately.

See MCP.md for full tool schemas and the privacy model.

Architecture

Bench runs entirely on Cloudflare. Each product is doing a specific job:

Code

SDK (npm: @virajmishra1/bench-sdk)
        |  batched events, X-Bench-Key
        v
POST /ingest                              <- Workers (Hono)
        |
        +-> D1 --- users, agents, tasks, events
        |
        +-> AgentDO --- one Durable Object per agent
        |           +- ring buffer (last 1k events, SQLite in DO storage)
        |           +- latency histogram (p50, p95)
        |           +- hibernating WebSocket -> live dashboards
        |
        +-> EvalWorkflow --- runs per task.end
        |           +- Workers AI (Llama 3.3 70B) -> score 0-1 + reasoning
        |              -> writes back to D1.tasks
        |              -> updates agents.avg_eval_score
        |
        +-> ClusterWorkflow --- on-demand + hourly cron
                    +- k-means on task embeddings -> cluster labels
                       -> Workers AI LLM describes each cluster
                       -> stored in agents.failure_clusters

Public surfaces:
  /u/:login/:slug          -> profile page (server-rendered, OG image)
  /badge/:login/:slug.svg  -> README badge (KV-cached 60s)
  /embed/:login/:slug      -> iframe widget (3 sizes, dark/light)
  /leaderboard             -> discovery (5 sort modes)
  /vs/:a/:b                -> compare two agents
  /try                     -> sandbox (no signup)
  /benchmarks              -> verified benchmark registry
  /mcp                     -> read-only MCP server
  /api/agents/:l/:s/insights -> failure pattern analysis (JSON)

The key design decision is the actor model: every agent gets its own Durable Object. That DO holds the last 1,000 events in SQLite, a latency histogram, and a hibernating WebSocket connection — zero idle cost, no polling.

Cloudflare products used

Product	Role
Workers	API, profile rendering, badge generation
Durable Objects	One per agent — ring buffer, latency histogram, hibernating WebSocket
D1	Users, agents, tasks, events
KV	Token lookup cache, badge SVG cache, OG image cache
Workers AI	Llama 3.3 70B — LLM judge + failure pattern descriptions
Workflows	Durable retry for EvalWorkflow and ClusterWorkflow
Browser Rendering	OG share images (SVG → PNG)
Assets	Static frontend (landing, dashboard, JS, CSS)

SDK reference

server.ts

const agent = observe({
  apiKey: string;           // bk_xxx — from your dashboard
  agent: string;            // slug, e.g. "my-agent"
  displayName?: string;
  endpoint?: string;        // default: bench.virajmishratakehome.workers.dev
  flushIntervalMs?: number; // default: 2000
  maxBatchSize?: number;    // default: 50
});

// Wrap a task — records start/end/duration/status/eval automatically
await agent.task("name", input, async (task) => {
  task.log("label", value);    // attach a log event
  task.cost(0.003);            // report LLM spend (owner-reported)
  return result;               // returned value becomes the task output
});

// Fire a custom event
agent.event("custom", { key: "value" });

// Flush immediately (auto-runs on batch full or interval)
await agent.flush();

task.cost() calls are labeled "owner-reported" in the UI. Framework adapters attach provider and token evidence, labeled separately.

Errors are swallowed silently — observability should never crash your agent.

Self-host

bash

git clone https://github.com/VirajMishra1/bench
cd bench && npm install

cd packages/worker
npx wrangler login

# Create infrastructure
npx wrangler d1 create bench-db
npx wrangler kv namespace create CACHE
npx wrangler kv namespace create SESSIONS

# Paste the returned IDs into wrangler.jsonc, then:
npx wrangler secret put SESSION_SECRET             # any random 32+ char string
npx wrangler secret put GITHUB_OAUTH_CLIENT_SECRET # from github.com/settings/developers

# Apply schema and deploy
npm run db:remote
npm run deploy

File layout

Code

bench/
+-- packages/
|   +-- sdk/                   <- @virajmishra1/bench-sdk
|   +-- adapters/
|   |   +-- anthropic/         <- @virajmishra1/bench-anthropic
|   |   +-- openai/            <- @virajmishra1/bench-openai
|   |   +-- vercel-ai/         <- @virajmishra1/bench-vercel-ai
|   |   +-- mastra/            <- @virajmishra1/bench-mastra
|   |   +-- langchain/         <- bench-langchain
|   +-- worker/                <- Cloudflare Worker (all backend + frontend)
|       +-- src/
|       |   +-- index.ts       <- Hono routes
|       |   +-- ingest.ts      <- POST /ingest
|       |   +-- profile.ts     <- public profile page
|       |   +-- badge.ts       <- SVG README badge
|       |   +-- embed.ts       <- iframe widget
|       |   +-- leaderboard.ts <- discovery page
|       |   +-- compare.ts     <- /vs/:a/:b
|       |   +-- do/agent.ts    <- AgentDO (actor per agent)
|       |   +-- workflows/
|       |       +-- eval.ts    <- LLM judge per task
|       |       +-- cluster.ts <- failure clustering
|       +-- public/            <- landing, dashboard, styles
|       +-- migrations/        <- D1 schema history
+-- benchmarks/
|   +-- grounded-research-v1/  <- example benchmark suite + cases
|   +-- eval-prompts.md        <- open-source judge prompts
+-- examples/                  <- runnable example agents

License

MIT — see LICENSE

Built by @virajm1shra on Cloudflare.