# Bench Agent Discovery

**Category:** 💻 Developer Tools  
**Repository:** https://github.com/VirajMishra1/bench  
**Views:** 0  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/bench-agent-discovery

## Description
Discover public AI agents, reusable recipes, and trusted benchmark evidence by task.

## Claude Desktop Quick Installation
Heuristic fallback — verify the package name and runner against the repository README before running it. Uses `npx` (confidence: low):

```json
"mcpServers": {
  "bench-agent-discovery": {
    "command": "npx",
    "args": ["-y","bench-agent-discovery"]
  }
}
```

## Documentation & README

<div align="center">

# bench

**See what the best agents do differently.**

One line of code. Live dashboard, public profile, README badge.

[![Live](https://img.shields.io/badge/live-bench.virajmishratakehome.workers.dev-e50914?style=for-the-badge)](https://bench.virajmishratakehome.workers.dev)
[![npm](https://img.shields.io/npm/v/%40virajmishra1%2Fbench-sdk?style=for-the-badge&label=npm)](https://www.npmjs.com/package/@virajmishra1/bench-sdk)
[![PyPI](https://img.shields.io/pypi/v/bench-observe?style=for-the-badge&label=PyPI)](https://pypi.org/project/bench-observe/)
[![License](https://img.shields.io/badge/license-MIT-22c55e?style=for-the-badge)](./LICENSE)
[![Built on](https://img.shields.io/badge/built_on-Cloudflare-f38020?style=for-the-badge)](https://workers.cloudflare.com/)

<img src="./assets/hero.svg" alt="Bench dashboard — live event stream, task history, eval scores, README badge" width="100%"/>

</div>

---

## What is this?

You built an AI agent. You ran it a few times. But you have no idea if it's actually working well — which tasks fail silently, what it costs per run, or how it compares to anything else.

Bench fixes that. Wrap your agent with one function call. You get:

- A public profile page showing runs, success rate, cost, and latency
- An auto-score on every task (0–1, LLM-as-judge)
- AI-generated summaries of your failure patterns
- A README badge that stays live and updates as your agent runs
- A public leaderboard so anyone can discover your agent

It's like GitHub for agents — observable, shareable, and public by default.

Want to see it before signing up? Try the sandbox at [/try](https://bench.virajmishratakehome.workers.dev/try) — no signup needed.

---

## Setup (3 minutes)

Sign in at [bench.virajmishratakehome.workers.dev](https://bench.virajmishratakehome.workers.dev) with GitHub. The dashboard gives you a copyable setup bundle — install command, API key, and first task template. It listens for your first event and links straight to your profile when it arrives.

Or do it manually:

```bash
npm install @virajmishra1/bench-sdk
export BENCH_KEY="bk_..."
```

```typescript
import { observe } from "@virajmishra1/bench-sdk";

const agent = observe({ apiKey: process.env.BENCH_KEY, agent: "my-agent" });

await agent.task("search", { query }, async (t) => {
  const result = await doSearch(query);
  t.log("found", result.length);
  t.cost(0.004);
  return result;
});
```

That's the whole SDK. Everything else is optional.

**Python:**

```bash
pip install bench-observe
export BENCH_KEY="bk_..."
```

```python
import bench

agent = bench.observe(api_key=os.environ["BENCH_KEY"], agent="my-agent")

async with agent.task_ctx("search", {"query": query}) as task:
    result = await do_search(query)
    task.log("found", len(result))
    task.set_output(result)
```

**Already on OpenTelemetry?** Point your exporter at Bench instead:

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT=https://bench.virajmishratakehome.workers.dev
export OTEL_EXPORTER_OTLP_PROTOCOL=http/json
export OTEL_EXPORTER_OTLP_HEADERS="X-Bench-Key=bka_...,X-Bench-Agent=my-agent"
```

Bench understands standard `gen_ai.*` spans — `invoke_agent`, `execute_tool`, `chat`, `retrieval`, and more.

**Prefer a CLI?** The stack-detecting CLI auto-instruments OpenAI, Anthropic, Vercel AI SDK, Mastra, and LangChain:

```bash
npx @virajmishra1/bench-cli init --install
npx @virajmishra1/bench-cli login
```

---

## What you get

| Feature | Description |
|---|---|
| **Live dashboard** | Real-time event stream while your agent runs. WebSocket, zero polling. |
| **Public profile** | `/u/you/your-agent` — shareable, OG-image ready, server-rendered |
| **README badge** | Live SVG badge. Updates automatically. GitHub camo-friendly. |
| **LLM eval** | Every task auto-scored 0–1 by a Llama 3.3 70B judge. Score logic is [open](./benchmarks/eval-prompts.md). |
| **Failure insights** | k-means clustering + LLM description of what keeps going wrong |
| **Leaderboard** | Browse public agents by runs, success rate, eval score, or cost |
| **Compare** | `/vs/@a/agent1/@b/agent2` — side-by-side quality, cost, latency |
| **Benchmarks** | Versioned benchmark suites with repeated runs and evidence trails. Separate from self-reported telemetry. |
| **MCP discovery** | Public read-only MCP server — `search_agents`, `get_agent`, `list_benchmarks` |
| **Embed widget** | `<iframe>`-ready mini-dashboard, 3 sizes, dark/light |
| **Privacy controls** | Hide inputs/outputs, make agents private, per-key access |
| **Permissioned reuse** | Publish capabilities with deny-by-default policies and quotas |

---

## Framework adapters

Drop-in wrappers that auto-instrument your existing LLM calls:

```typescript
// Anthropic — wraps every messages.create() call
import { wrapAnthropic } from "@virajmishra1/bench-anthropic";
const client = wrapAnthropic(new Anthropic(), bench);

// OpenAI — wraps chat completions, responses, and embeddings
import { wrapOpenAI } from "@virajmishra1/bench-openai";
const client = wrapOpenAI(new OpenAI(), bench);

// Vercel AI SDK — wraps generateText / streamText / generateObject
import { track } from "@virajmishra1/bench-vercel-ai";
const result = await track(bench, "summarize", () =>
  generateText({ model: anthropic("claude-sonnet-4-6"), prompt: "..." })
);

// Mastra
import { wrapMastra } from "@virajmishra1/bench-mastra";
```

---

## Let your AI find agents

Bench exposes a public MCP server at `/mcp`. Connect it to Claude Code:

```bash
claude mcp add --transport http bench https://bench.virajmishratakehome.workers.dev/mcp
```

Or Codex:

```bash
codex mcp add bench --url https://bench.virajmishratakehome.workers.dev/mcp
```

Tools available: `search_agents`, `get_agent`, `list_benchmarks`. Search returns only public agents. Owner telemetry and benchmark evidence are labeled separately.

See [MCP.md](./MCP.md) for full tool schemas and the privacy model.

---

## Architecture

Bench runs entirely on Cloudflare. Each product is doing a specific job:

```
SDK (npm: @virajmishra1/bench-sdk)
        |  batched events, X-Bench-Key
        v
POST /ingest                              <- Workers (Hono)
        |
        +-> D1 --- users, agents, tasks, events
        |
        +-> AgentDO --- one Durable Object per agent
        |           +- ring buffer (last 1k events, SQLite in DO storage)
        |           +- latency histogram (p50, p95)
        |           +- hibernating WebSocket -> live dashboards
        |
        +-> EvalWorkflow --- runs per task.end
        |           +- Workers AI (Llama 3.3 70B) -> score 0-1 + reasoning
        |              -> writes back to D1.tasks
        |              -> updates agents.avg_eval_score
        |
        +-> ClusterWorkflow --- on-demand + hourly cron
                    +- k-means on task embeddings -> cluster labels
                       -> Workers AI LLM describes each cluster
                       -> stored in agents.failure_clusters

Public surfaces:
  /u/:login/:slug          -> profile page (server-rendered, OG image)
  /badge/:login/:slug.svg  -> README badge (KV-cached 60s)
  /embed/:login/:slug      -> iframe widget (3 sizes, dark/light)
  /leaderboard             -> discovery (5 sort modes)
  /vs/:a/:b                -> compare two agents
  /try                     -> sandbox (no signup)
  /benchmarks              -> verified benchmark registry
  /mcp                     -> read-only MCP server
  /api/agents/:l/:s/insights -> failure pattern analysis (JSON)
```

The key design decision is the **actor model**: every agent gets its own Durable Object. That DO holds the last 1,000 events in SQLite, a latency histogram, and a hibernating WebSocket connection — zero idle cost, no polling.

### Cloudflare products used

| Product | Role |
|---|---|
| **Workers** | API, profile rendering, badge generation |
| **Durable Objects** | One per agent — ring buffer, latency histogram, hibernating WebSocket |
| **D1** | Users, agents, tasks, events |
| **KV** | Token lookup cache, badge SVG cache, OG image cache |
| **Workers AI** | Llama 3.3 70B — LLM judge + failure pattern descriptions |
| **Workflows** | Durable retry for EvalWorkflow and ClusterWorkflow |
| **Browser Rendering** | OG share images (SVG → PNG) |
| **Assets** | Static frontend (landing, dashboard, JS, CSS) |

---

## SDK reference

```typescript
const agent = observe({
  apiKey: string;           // bk_xxx — from your dashboard
  agent: string;            // slug, e.g. "my-agent"
  displayName?: string;
  endpoint?: string;        // default: bench.virajmishratakehome.workers.dev
  flushIntervalMs?: number; // default: 2000
  maxBatchSize?: number;    // default: 50
});

// Wrap a task — records start/end/duration/status/eval automatically
await agent.task("name", input, async (task) => {
  task.log("label", value);    // attach a log event
  task.cost(0.003);            // report LLM spend (owner-reported)
  return result;               // returned value becomes the task output
});

// Fire a custom event
agent.event("custom", { key: "value" });

// Flush immediately (auto-runs on batch full or interval)
await agent.flush();
```

`task.cost()` calls are labeled "owner-reported" in the UI. Framework adapters attach provider and token evidence, labeled separately.

Errors are swallowed silently — observability should never crash your agent.

---

## Self-host

```bash
git clone https://github.com/VirajMishra1/bench
cd bench && npm install

cd packages/worker
npx wrangler login

# Create infrastructure
npx wrangler d1 create bench-db
npx wrangler kv namespace create CACHE
npx wrangler kv namespace create SESSIONS

# Paste the returned IDs into wrangler.jsonc, then:
npx wrangler secret put SESSION_SECRET             # any random 32+ char string
npx wrangler secret put GITHUB_OAUTH_CLIENT_SECRET # from github.com/settings/developers

# Apply schema and deploy
npm run db:remote
npm run deploy
```

---

## File layout

```
bench/
+-- packages/
|   +-- sdk/                   <- @virajmishra1/bench-sdk
|   +-- adapters/
|   |   +-- anthropic/         <- @virajmishra1/bench-anthropic
|   |   +-- openai/            <- @virajmishra1/bench-openai
|   |   +-- vercel-ai/         <- @virajmishra1/bench-vercel-ai
|   |   +-- mastra/            <- @virajmishra1/bench-mastra
|   |   +-- langchain/         <- bench-langchain
|   +-- worker/                <- Cloudflare Worker (all backend + frontend)
|       +-- src/
|       |   +-- index.ts       <- Hono routes
|       |   +-- ingest.ts      <- POST /ingest
|       |   +-- profile.ts     <- public profile page
|       |   +-- badge.ts       <- SVG README badge
|       |   +-- embed.ts       <- iframe widget
|       |   +-- leaderboard.ts <- discovery page
|       |   +-- compare.ts     <- /vs/:a/:b
|       |   +-- do/agent.ts    <- AgentDO (actor per agent)
|       |   +-- workflows/
|       |       +-- eval.ts    <- LLM judge per task
|       |       +-- cluster.ts <- failure clustering
|       +-- public/            <- landing, dashboard, styles
|       +-- migrations/        <- D1 schema history
+-- benchmarks/
|   +-- grounded-research-v1/  <- example benchmark suite + cases
|   +-- eval-prompts.md        <- open-source judge prompts
+-- examples/                  <- runnable example agents
```

---

## License

MIT — see [LICENSE](./LICENSE)

Built by [@virajm1shra](https://x.com/virajm1shra) on Cloudflare.