Agent Web

URL to LLM-ready markdown - a polite, robots-respecting web reader for agents. Free.

Quick Install

Automated & IDE Setup

Copy the AI prompt to install this server into Claude Code, Cursor, or another agent — or use 1-click editor setup below.

Add to Cursor Add to VS Code

Manual Client & Custom JSON ConfigExpand JSON ▾

Client Config & Setup

Remote HTTP

Choose your client or environment

Target File:~/Library/Application Support/Claude/claude_desktop_config.json

claude_desktop_config.json

{
  "mcpServers": {
    "agent-web": {
      "url": "https://agent-web.foomworks.workers.dev**"
    }
  }
}

💡 Paste the JSON block into your client's configuration file under mcpServers, then restart the application.

Documentation Overview

agent-web

URL → LLM-ready markdown. A charter-clean, polite, robots-respecting web reader for AI agents — honors robots.txt, identifies honestly, and never bypasses anti-bot/CAPTCHA or paywalls. No account, no API key — free per-call access.

🌐 Live: https://agent-web.foomworks.workers.dev · 🔌 MCP: https://agent-web.foomworks.workers.dev/mcp

What it gives an agent

Give it one publicly reachable URL; get back the page as clean, LLM-ready markdown — title, word count, and the text — without parsing raw HTML yourself. Server-rendered (static) HTML pages. Ideal for summarizing an article, extracting docs, or feeding a page into an LLM.

Connect as an MCP server (recommended)

Streamable HTTP, JSON-RPC 2.0:

Code

https://agent-web.foomworks.workers.dev/mcp

Tool	Cost	Returns
`read_url`	free	fetch a URL → full LLM-ready markdown (title, word count, markdown)
`read_url_preview`	free	the first ~600 characters of the markdown — a cheap look before the full pull

Or call it over HTTP

bash

BASE=https://agent-web.foomworks.workers.dev
curl -s "$BASE/read?url=https://example.com/&ref=readme"          # full markdown
curl -s "$BASE/read/preview?url=https://example.com/&ref=readme"  # first ~600 chars

Discovery: /.well-known/mcp.json (MCP) · /.well-known/agent-card.json (A2A) · /openapi.json · /policy (Acceptable-Use Policy)

Charter-clean by design (the bright line)

This is a polite reader, not a scraper:

Honors the origin's robots.txt for our user-agent — a disallowed path is refused without fetching; an unconfirmable robots.txt (5xx/error) is treated conservatively as disallowed.
Identifies honestly with a descriptive user-agent — never spoofs a browser.
Read-only GET of the single URL you supply — no crawl, no proxy rotation.
Never bypasses anti-bot/CAPTCHA, paywalls, or login walls.
SSRF-guarded: private/loopback/link-local/internal hosts are blocked and every redirect hop is re-validated.

You are responsible for your right to fetch the URL you submit.

Limits

Static HTML only for now — JavaScript-rendered pages, screenshots, and PDF are a later increment.
Body cap ~2 MB, fetch timeout ~12 s.
The service holds no keys and never initiates payments.

Install the AgentSkill

SKILL.md is a portable AgentSkill (ClawHub / skills.sh / Claude Skills).

About

agent-web is operated by FOOM — an AI-operated, human-supervised autonomous service. License: MIT.

Read the full README →View source on GitHub →

Related MCP Servers

View all in Developer Tools View all alternatives

Reviews

No reviews yet — be the first to share how this listing worked for you.

Frequently Asked Questions about Agent Web

Add the following block to your claude_desktop_config.json under mcpServers: "mcpServers": { "agent-web": { "command": "npx", "args": ["-y", "agent-web"] } }

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:

Live Dynamic SVG Preview

Markdown (GitHub README)

[![AllMCPs](https://allmcps.com/api/badge/agent-web?style=directory)](https://allmcps.com/mcp/agent-web)

HTML Embed

<a href="https://allmcps.com/mcp/agent-web"><img src="https://allmcps.com/api/badge/agent-web?style=directory" alt="Agent Web on AllMCPs" /></a>

Technical Specs & Signals

CategoryDeveloper Tools

36Quality signal: Fair · 36/100How this signal is calculated ▾

Server availabilityNot measured

Not scored for repo-hosted servers — we can't reach the running server, only its GitHub page. Hosted MCP endpoints are health-checked live.

Verified ownership10/20

Documentation & tools16/30

Adoption & activity1/15

Community engagement0/10

A guidance signal from public completeness & health data — not a user rating. New listings start lower and rise as they add docs, get verified, and grow adoption. Signals we can't observe for a listing are skipped, not counted against it.

★ Featured

Moxie Docs MCP

MCP & Agent Skills for Automated Documentation, and codebase conventions + context

Explore Server →

Own this project?

This directory is pre-filled from public sources. Claim via GitHub README, site badge, or DNS TXT to unlock edit access and the Official badge — proof is checked automatically, then reviewed by our team.

Free dofollow backlink: add your website and place the AllMCPs badge on it — no claim needed. We detect it automatically and keep it verified as long as the badge stays live.

Claim & get free dofollow

Share & Embed

Add our SVG badge (dark/light directory styles) or embeddable widget to your site.

Explore more

More in 💻 Developer Tools →Best MCP servers for Developers →Alternatives to Agent Web →Install in Claude Desktop Install in Cursor Install in VS Code