Log inSubmit server
Webclaw logo

Webclaw

by 0xMassi

Extracts, searches, compares, and researches web content through MCP tools with markdown, JSON, screenshots, and LLM-ready output.

InstallRepositoryWebsite
Full README Compare alternatives Embed widget Claim this listing

Install Webclaw

Pick your client, then add it in one click or paste the config.

STDIONode.js
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "0xmassi-webclaw": {
      "command": "npx",
      "args": [
        "-y",
        "create-webclaw"
      ]
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

Overview

0xMassi/webclaw MCP server gives AI agents tools for extracting and analyzing web pages, including scraping, crawling, searching, structured extraction, screenshots, and research. It can return markdown, text, JSON, LLM-oriented content, HTML, images, PDFs, or MHTML, while TLS fingerprinting helps access sites without a browser. Use it for web research, RAG pipelines, page monitoring, and targeted data collection when raw HTML is too noisy. The `npx create-webclaw` installer configures supported MCP clients including Claude Desktop, Claude Code, Cursor, Windsurf, and Codex CLI.

Use cases

  • Scrape documentation into clean LLM-ready context
  • Extract product records from pages with CSS selectors or XPath
  • Crawl a website and prioritize pages relevant to a query
  • Compare page snapshots to detect content changes
  • Research a topic across multiple web sources

Key features

  • Returns markdown, text, JSON, HTML, and LLM-oriented page content
  • Extracts records, attributes, tables, and text with CSS or XPath
  • Discovers URLs through crawling, sitemaps, and Common Crawl
  • Captures screenshots, PDFs, and MHTML files
  • Searches the web and supports multi-source research
  • Uses TLS fingerprinting to bypass some anti-bot defenses without a browser

Tools12

What Webclaw lets your agent do. Expand a tool for its parameters.

Self-reported~199 tokens of context

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked Today

Recent health check succeeded.

96/96 checks healthy over the last 41d

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 5d ago via OSV.dev

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars2.4kStargazers on the source repository.
Last commit1d agoMost recent push to the default branch.
Tools exposed12Callable tools this server registers over MCP.
Directory activity2 installsConfig copies, upvotes, and views on AllMCPs.

How Webclaw works

What 0xMassi/webclaw MCP server does

0xMassi/webclaw turns web pages into formats that agents and downstream applications can consume. Its scraping workflow can return cleaned markdown, plain text, JSON, LLM-oriented content, or the original HTML. The server also supports targeted extraction with CSS selectors, XPath, page tables, text matches, or repeated elements modeled on an example record.

Beyond single-page retrieval, it provides tools for finding and processing related content. Crawling follows same-origin links and prioritizes pages using a query. URL mapping can inspect sitemaps, perform a short crawl, and consult the Common Crawl index without extracting every discovered page. Batch processing scrapes multiple URLs in parallel.

Other tools cover higher-level web tasks: structured-data extraction, page summarization, snapshot comparison, brand asset discovery, web search with result scraping, and multi-source research. Screenshot output can capture the viewport or full page as PNG or JPEG, create a PDF, or save a single MHTML file.

How it works

The MCP interface exposes web retrieval and transformation operations to compatible AI clients. Pages that require JavaScript are rendered before extraction. The select workflow returns matched values or records as JSON; links in href and src are converted to absolute URLs. XPath expressions can return element text, text-node content, attribute values, or scalar results such as count() and string().

For structured records, fields are evaluated within each matched element and produce the first value or null. Table extraction returns captions when available, column headers, and rows of cell text. The system also uses TLS fingerprinting to bypass some anti-bot defenses without launching a browser.

Setup and configuration

The quickest client setup is:

Terminal
npx create-webclaw

The installer detects supported clients and writes the MCP configuration. Named integrations include Claude Code, Claude Desktop, Cursor, Windsurf, OpenCode, and Codex CLI.

Other installation paths include Homebrew, prebuilt macOS, Linux, and Windows binaries, Docker, and Cargo packages for the CLI and MCP server. A Docker invocation is available with:

Terminal
docker run --rm ghcr.io/0xmassi/webclaw https://example.com

The project can also be built as a Python wheel. Its Python wrapper bundles the binary and exposes scraping, selection, table, text-matching, similarity, crawling, mapping, and screenshot functions without Python dependencies or an API key.

Tools and capabilities

  • scrape: Retrieve one URL in markdown, text, JSON, LLM, or HTML form.
  • select: Extract values, records, tables, text-matched elements, or repeated structures with CSS selectors or XPath.
  • screenshot: Capture viewport or full-page images, PDFs, or MHTML.
  • crawl: Follow same-origin links and extract relevant pages.
  • map: Discover URLs from sitemaps, short crawls, and Common Crawl.
  • batch: Process several URLs concurrently.
  • extract: Convert page content into structured data.
  • summarize: Produce a page summary.
  • diff: Compare content snapshots.
  • brand: Find colors, fonts, logos, and metadata.
  • search: Search the web and scrape result pages.
  • research: Run a multi-source research workflow.

Limitations and notes

Crawling is restricted to same-origin links. Selector-based extraction depends on the target page's HTML structure, while pages requiring JavaScript are rendered before extraction. Windows source builds require platform tools including Rust, CMake, LLVM, NASM, Visual Studio Build Tools, and the Windows SDK; prebuilt binaries provide a simpler route. The Windows native installation guidance covers x64 binaries rather than ARM64.

Read the full README View source on GitHub

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about Webclaw

0xMassi/webclaw is an MCP server that connects AI agents to web pages for scraping, selection, crawling, mapping, batch retrieval, structured extraction, search, summarization, screenshots, and research. It returns clean markdown, text, JSON, HTML, images, PDFs, or LLM-oriented content, with XPath and CSS selector support for targeted data.

Run `npx create-webclaw` to detect supported clients and configure the server automatically. You can also install it with Homebrew, Docker, Cargo, or a prebuilt binary for macOS, Linux, or Windows.

Yes, 0xMassi/webclaw works with Cursor. The `npx create-webclaw` installer detects Cursor and configures the MCP server for it.

Yes, 0xMassi/webclaw can extract structured records with CSS selectors or XPath. Its selection workflow can return attributes, element text, HTML, page tables, or fields evaluated within each matched record as JSON.

Related MCP servers

All in Search & Data Extraction All alternatives
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewWebclaw AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/0xmassi-webclaw?style=directory)](https://allmcps.com/mcp/0xmassi-webclaw)
HTML Embed
<a href="https://allmcps.com/mcp/0xmassi-webclaw"><img src="https://allmcps.com/api/badge/0xmassi-webclaw?style=directory" alt="Webclaw on AllMCPs" /></a>