Commoncrawl
Search Common Crawl snapshots, inspect archived URL captures, and retrieve page content from WARC records.
Install Commoncrawl
Pick your client, then add it in one click or paste the config.
~/Library/Application Support/Claude/claude_desktop_config.json{
"mcpServers": {
"commoncrawl": {
"command": "npx",
"args": [
"-y",
"@pipeworx/mcp-commoncrawl"
]
}
}
}
Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.
Install not yet sandbox-tested— the server itself is confirmed live (we called its tools/list endpoint). The stdio install command is a separate, ongoing check.
Overview
Use cases
- Find captures of a URL during a specified month
- Compare archived versions of a page
- Verify whether Common Crawl saw a domain
- Retrieve page content after a site went offline
- Filter archived captures by status or date
Key features
- Lists monthly Common Crawl collections
- Searches CDX indexes by URL, host, or domain
- Retrieves WARC records by byte range
- Returns capture and HTTP metadata
- Supports pagination, filters, and date ranges
- Provides hosted HTTP and local stdio access
Tools3
What Commoncrawl lets your agent do. Expand a tool for its parameters.
Security & health
What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.
Recent health check succeeded.
What it can do
From the 3 tools this server returned when we ran it; 3 declare their own safety hints.
Observed 2026-11-02 in an isolated sandbox. A description of reach, not a security audit.
Supply-chain signal
No high-severity advisories surfaced by our automated scan.
Scanned 7d ago via OSV.dev · @pipeworx/mcp-commoncrawl (npm)
Adoption & maintenance
Factual signals from GitHub, npm, and our automated checks — not a rating.
How Commoncrawl works
Written by AllMCPs with AI assistance from the project’s README and listing data. How we build listings
Reviews
No reviews yet — be the first to share how it worked for you.
Frequently Asked Questions about Commoncrawl
Related MCP servers
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.
AllMCPs Directory Badge
Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.
[](https://allmcps.com/mcp/commoncrawl)<a href="https://allmcps.com/mcp/commoncrawl"><img src="https://allmcps.com/api/badge/commoncrawl?style=directory" alt="Commoncrawl on AllMCPs" /></a>