X402 Scraper API

What x402-scraper-api does

The x402-scraper-api MCP server gives MCP clients three web-scraping operations with different extraction methods and prices. scrape_basic parses static HTML and returns cleaned Markdown for fast, relatively simple pages. scrape_dynamic uses headless browser rendering for sites that depend on JavaScript or other dynamic behavior. scrape_structured extracts page content into a JSON shape supplied by the caller through a requested JSON Schema.

The tools are monetized through HTTP 402 payment challenges. Charges are listed in USDC on Base Mainnet: $0.01 for basic scraping, $0.05 for dynamic scraping, and $0.15 for structured extraction. The service is relevant to the Search & Data Extraction category, particularly for autonomous agents, research pipelines, and LLM workflows that need to retrieve data from web pages programmatically.

How it works

For local use, an MCP client starts the package with npx -y x402-scraper-api. The supplied configuration sets BASE_RPC_URL to a Base network RPC endpoint. Once connected, the client can discover and call the scraping tools through MCP.

The hosted deployment also exposes an MCP gateway at /mcp. Agents can send standard JSON-RPC requests to that endpoint with a tool name and arguments such as a target URL and, for structured extraction, a jsonSchema object. The deployment publishes a server card at /.well-known/mcp/server-card.json, allowing autonomous agents to discover its tool definitions. A separate /telemetry endpoint returns production metrics that include request velocity, active sessions, and revenue milestones.

The x402-scraper-api MCP server requires payment handling appropriate to the selected tool. The provided material identifies Base Mainnet and USDC as the payment network and currency, but does not document wallet configuration or payment-client setup.

Setup and configuration

Add the server to an MCP client configuration such as claude_desktop_config.json:

{
  "mcpServers": {
    "x402-scraper": {
      "command": "npx",
      "args": ["-y", "x402-scraper-api"],
      "env": {
        "BASE_RPC_URL": "https://mainnet.base.org"
      }
    }
  }
}

The README specifically documents Claude Desktop and refers generally to MCP clients. For direct HTTP use, send JSON-RPC over POST to https://x402-scraper-api-production-67a4.up.railway.app/mcp with Content-Type: application/json.

Tools and capabilities

• scrape_basic: Parses static HTML and returns clean Markdown; listed price: $0.01.
• scrape_dynamic: Renders JavaScript-heavy pages with a headless browser; listed price: $0.05.
• scrape_structured: Maps scraped page content to a caller-provided JSON Schema; listed price: $0.15.
• Server-card discovery through the hosted .well-known endpoint.
• Telemetry retrieval through the hosted /telemetry endpoint.

Limitations and notes

The service is paid rather than an unmetered local scraper. The listed prices apply per tool tier and are denominated in USDC on Base Mainnet. Basic scraping is intended for static HTML, while dynamic scraping is the documented option for JavaScript-heavy pages. The README does not specify supported websites, crawl limits, authentication handling, proxy behavior, retries, response-size limits, or data-retention policies. It also does not describe additional environment variables beyond the Base RPC URL shown in the setup example.

The x402-scraper-api MCP server is distributed under the MIT license. Its hosted endpoints include a production MCP gateway, server-card URL, and telemetry URL, but the README does not provide availability guarantees for those endpoints.