MCP Read Website Fast

What the just-every/mcp-read-website-fast MCP server does

The just-every/mcp-read-website-fast MCP server retrieves web pages locally and extracts their primary content instead of returning the surrounding page noise. It converts the result to Markdown and keeps links in the output, which is useful when an agent needs references or wants to build link-aware knowledge structures.

Content extraction uses Mozilla Readability, the same general approach associated with Firefox Reader View. HTML is converted with Turndown and GitHub-Flavored Markdown support. The server is intended for Claude Code, IDE integrations, and other MCP-based LLM workflows where reducing the amount of content sent to a model matters.

How it works

The read_website tool requires a URL and accepts an optional pages value. A single page is the default, while the maximum supported value is 100. When more than one page is requested, the crawler can fetch pages concurrently and apply a configurable crawl depth through the underlying implementation.

Requests are handled with robots.txt support and rate limiting by default. URLs are cached on disk using SHA-256-derived keys. The implementation uses a stream-first approach to limit memory usage, and optional chunking is available for downstream processing. Sites that block automated requests or require JavaScript may not produce extracted content; the README specifically notes that JavaScript-required sites are not supported.

Setup and configuration

Install the just-every/mcp-read-website-fast MCP server with npm and run it through npx:

npx -y @just-every/mcp-read-website-fast

The README provides MCP configuration examples for Claude Code, VS Code, Cursor, JetBrains IDEs, and generic MCP client JSON. No API key or other credential is listed as required. The server also includes an automatic restart wrapper that retries after crashes with exponential backoff, up to 10 attempts within one minute. Development commands can run fetches directly, build the project, or clear the cache.

Command-line configuration includes page count, concurrency, request timeout, user agent, cache directory, output format, robots.txt behavior, and whether cross-origin crawling is allowed. These options apply to the development or command-line interface documented by the project.

Tools and capabilities

The just-every/mcp-read-website-fast MCP server exposes:

• read_website: Fetches a webpage and returns clean Markdown. It accepts a required url and optional pages limit.
• read-website-fast://status: Reports cache statistics.
• read-website-fast://clear-cache: Clears the cache directory.

The project also documents JSON, Markdown, and combined URL-plus-Markdown output modes for its command-line usage. Markdown is the default format.

Limitations and notes

Some websites may block automated access, time out, or return little usable content after extraction. The documented remedies include increasing the timeout, checking network access, and supplying a custom user agent. JavaScript-dependent sites are not supported according to the troubleshooting notes.

The server honors robots.txt unless that behavior is explicitly disabled through the command-line option. Its license is MIT, and the README identifies @just-every/crawl as the package used for core crawling and Markdown conversion.