404.directory

Risk preflight for AI Agent actions.

404.directory gives AI Agents an evidence-backed allow, review, or block decision before they install or invoke a third-party tool. Discovery, provider verification, live checks, privacy-safe usage evidence, and a curated read-only MCP gateway support that decision.

The first vertical decision workflow evaluates Polymarket settlement wording, timing, public order-book liquidity, caller-observed eligibility, and execution mode before an Agent contemplates a Yes/No action. It never predicts the winner or places an order.

Connect a real Agent in under a minute (Codex, Cursor, Claude Code, or MCP SDK): https://404.directory/connect?source=github

External users can join the first-10 activation pilot and report only the client, task category, and failure stage: https://github.com/MM-sheng/404-directory/issues/1

Agent-readable installation instructions: llms-install.md

Install the Agent Skill in Codex, Claude Code, Cursor, Cline, or another Agent Skills client:

npx skills add MM-sheng/404-directory --skill use-404-directory -g -y

The repository also conforms to Agent Plugins 1.0: compatible clients discover the Agent Skill from skills/ and an identity-preserving bridge to the hosted Streamable HTTP server from the root mcp.json. The bridge creates one random ID in the client-managed PLUGIN_DATA directory. The raw ID stays local; the service stores only an HMAC digest after a successful tool call.

Claude Code and Cowork use the native manifest in .claude-plugin/. It loads the same Skill and identity-preserving bridge with Claude's persistent plugin data directory, so updates keep the installation identity stable.

Install it directly in Claude Code while the official directory submission is under review:

/plugin marketplace add MM-sheng/404-directory
/plugin install 404-directory@404-directory

Product layers

The current product is intentionally narrow: preflight one prediction-market decision or one registered third-party tool action, then capture a bounded outcome. Future identity, reputation, guarantee, and insurance layers remain hypotheses until real external Agent usage validates them.

Current first-party tools

Agent Discovery API (/v1)

Requires a catalog backend (DATABASE_URL Postgres, or in-memory fallback when CATALOG_MEMORY_FALLBACK=true).

# Bootstrap admin token (required in production; auto-generated in local/dev)
export REGISTRY_ADMIN_TOKEN=change-me-to-a-long-secret

# Register a tool (pending quarantine until ownership + protocol verification)
curl -sS http://127.0.0.1:4040/v1/tools \
  -H 'content-type: application/json' \
  -H "authorization: Bearer $REGISTRY_ADMIN_TOKEN" \
  -d '{
    "name":"btc_analyzer",
    "description":"Analyze BTC market signals for agents",
    "capabilities":["btc","market-analysis"],
    "protocol":"mcp",
    "endpoint":"https://example.com/mcp",
    "category":"finance",
    "provider":{"name":"Example Labs","identity":{"type":"domain","value":"example.com"}}
  }'
# Response includes provider_api_key once — store it for ownership + further writes.

# Search (active tools only)
curl -sS 'http://127.0.0.1:4040/v1/tools/search?capability=btc&trust_threshold=0.2'

# Trust profile
curl -sS http://127.0.0.1:4040/v1/tools/btc_analyzer/trust

Trust Profile dimensions (v1 algorithm, extensible factors JSON):

• Ownership / Availability / Compatibility / Security / Usage → overall_score

Contextual preflight is available through POST /v1/evaluations; public receipts are readable at GET /v1/evaluations/:id. One bounded outcome can be attached through POST /v1/evaluations/:id/outcome using the one-time token returned at evaluation time. Only the token hash is stored, and self-reported outcomes never directly increase Trust. The older generic POST /v1/receipts remains disabled because unbound anonymous submissions would poison Trust.

Copy-ready Agent trigger policy and examples: docs/AGENT_RISK_PREFLIGHT.md

Privacy-safe product validation is public at GET /v1/metrics/risk-evaluations: evaluation volume, decision distribution, outcome-report rate, and behavior-change rate, without prompts or raw identity.

The prediction-market workflow is documented at docs/PREDICTION_MARKET_PREFLIGHT.md. Its privacy-safe aggregate metrics are available at GET /v1/metrics/prediction-market-evaluations.

MCP Discovery tools

When the catalog is enabled, MCP also exposes:

• evaluate_prediction_market
• report_prediction_market_outcome
• evaluate_tool_risk
• report_tool_outcome
• search_tools
• get_tool
• compare_tools
• get_trust_score
• recommend_tools
• list_capabilities
• get_capability_graph
• search_official_docs
• inspect_tool_server
• invoke_registered_tool

alongside the existing executable tools.

search_official_docs is the low-friction path: one call searches current first-party OpenAI, Microsoft Learn, AWS, and Cloudflare documentation in parallel, with source-level provenance and partial-failure reporting. For other curated servers, discover a catalog server, inspect its live allowlisted schemas, then invoke one approved tool. Arbitrary URLs, authenticated servers, non-active entries, unverified providers, and destructive tools are rejected. Remote results are bounded and explicitly marked as untrusted external data.

MCP task prompts

Clients that expose MCP Prompts also receive three task-oriented starting points:

• research-official-docs — turns a real technical question into a search_official_docs call;
• verify-public-deployment — turns a concrete public deployment claim into a verify_web call;
• evaluate-agent-tool — finds a catalog candidate, calls the contextual risk preflight, and requires an allow, review, or block result.

Rendering or opening a prompt never counts toward the 1,000-Agent target. Each template explicitly requires a non-error tool result that materially answers the user's task. The server records only aggregate prompts/list and prompts/get activation stages, never prompt arguments or task text.

Capability Graph

Agents can explore shared-capability edges and get related-tool recommendations:

curl -sS http://127.0.0.1:4040/v1/capabilities | jq
curl -sS http://127.0.0.1:4040/v1/graph/capabilities | jq '.edges[:3]'
curl -sS http://127.0.0.1:4040/v1/tools/verify_web/related | jq

Similarity is Jaccard over capability sets, with small boosts for matching protocol/category (cap_v1). This is the seed of the long-term Capability Graph.

Quick start

npm install
npx playwright install chromium
cp .env.example .env
npm run dev

Default: http://127.0.0.1:4040

With Postgres:

docker compose up -d postgres
export DATABASE_URL=postgres://404:404@127.0.0.1:5432/404
npm run db:migrate
npm run dev

On boot, first-party tools are seeded into the catalog (SEED_FIRST_PARTY_TOOLS=true) so GET /v1/tools/search?capability=web-verification returns verify_web. The six operator-reviewed public MCP servers are also seeded as pending entries when SEED_CURATED_MCP_SERVERS=true. The verification worker performs live MCP admission before they become discoverable or executable.

Verification worker

• Default: VERIFICATION_WORKER_MODE=inline (loop inside HTTP process)
• Split out for production load:

export VERIFICATION_WORKER_MODE=external
npm run worker:verify

Provider ownership (DNS TXT or GitHub bio)

# Domain provider
curl -sS -X POST http://127.0.0.1:4040/v1/providers/example-labs/ownership/challenge
# Publish DNS: _404-directory.example.com TXT "404-directory-verify=<token>"
curl -sS -X POST http://127.0.0.1:4040/v1/providers/example-labs/ownership/verify

Read the full README on GitHub →