# 0xMassi/webclaw [Health: Active]

**Category:** 🔎 Search & Data Extraction  
**Repository:** https://github.com/0xMassi/webclaw  
**GitHub Stars:** 2394  
**Views:** 13  
**Installs:** 2  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/0xmassi-webclaw

## Description
Web content extraction for AI agents. 10 tools: scrape, crawl, map, batch, extract, summarize, diff, brand, search, research. TLS fingerprinting bypasses anti-bot without a browser. 67% fewer tokens than raw HTML. npx create-webclaw auto-configures Claude, Cursor, Windsurf, Codex, OpenCode.

## Tools
Capabilities this server exposes over MCP:

- **scrape** — Extract one URL as markdown, text, JSON, LLM format, or HTML
- **select** — Pick values or records from a page with CSS selectors or XPath, its data tables, elements found by text, or every element like an example, as JSON
- **screenshot** — Capture a page's viewport or the whole page as PNG or JPEG, print it to PDF, or save it as one MHTML file
- **crawl** — Follow same-origin links and extract discovered pages, most relevant first with a `query
- **map** — Discover URLs from sitemaps, a short crawl and the Common Crawl index without extracting every page
- **batch** — Scrape multiple URLs in parallel
- **extract** — Convert page content into structured data
- **summarize** — Summarize a page
- **diff** — Compare page content snapshots
- **brand** — Extract colors, fonts, logos, and metadata
- **search** — Search the web and scrape results
- **research** — Multi-source research workflow

## Claude Desktop Quick Installation
Install path detected from listing signals. Uses `npx` (confidence: high):

```json
"mcpServers": {
  "webclaw": {
    "command": "npx",
    "args": ["-y","create-webclaw"]
  }
}
```

## Documentation

## What 0xMassi/webclaw MCP server does

0xMassi/webclaw turns web pages into formats that agents and downstream applications can consume. Its scraping workflow can return cleaned markdown, plain text, JSON, LLM-oriented content, or the original HTML. The server also supports targeted extraction with CSS selectors, XPath, page tables, text matches, or repeated elements modeled on an example record.

Beyond single-page retrieval, it provides tools for finding and processing related content. Crawling follows same-origin links and prioritizes pages using a query. URL mapping can inspect sitemaps, perform a short crawl, and consult the Common Crawl index without extracting every discovered page. Batch processing scrapes multiple URLs in parallel.

Other tools cover higher-level web tasks: structured-data extraction, page summarization, snapshot comparison, brand asset discovery, web search with result scraping, and multi-source research. Screenshot output can capture the viewport or full page as PNG or JPEG, create a PDF, or save a single MHTML file.

## How it works

The MCP interface exposes web retrieval and transformation operations to compatible AI clients. Pages that require JavaScript are rendered before extraction. The `select` workflow returns matched values or records as JSON; links in `href` and `src` are converted to absolute URLs. XPath expressions can return element text, text-node content, attribute values, or scalar results such as `count()` and `string()`.

For structured records, fields are evaluated within each matched element and produce the first value or `null`. Table extraction returns captions when available, column headers, and rows of cell text. The system also uses TLS fingerprinting to bypass some anti-bot defenses without launching a browser.

## Setup and configuration

The quickest client setup is:

```bash
npx create-webclaw
```

The installer detects supported clients and writes the MCP configuration. Named integrations include Claude Code, Claude Desktop, Cursor, Windsurf, OpenCode, and Codex CLI.

Other installation paths include Homebrew, prebuilt macOS, Linux, and Windows binaries, Docker, and Cargo packages for the CLI and MCP server. A Docker invocation is available with:

```bash
docker run --rm ghcr.io/0xmassi/webclaw https://example.com
```

The project can also be built as a Python wheel. Its Python wrapper bundles the binary and exposes scraping, selection, table, text-matching, similarity, crawling, mapping, and screenshot functions without Python dependencies or an API key.

## Tools and capabilities

- `scrape`: Retrieve one URL in markdown, text, JSON, LLM, or HTML form.
- `select`: Extract values, records, tables, text-matched elements, or repeated structures with CSS selectors or XPath.
- `screenshot`: Capture viewport or full-page images, PDFs, or MHTML.
- `crawl`: Follow same-origin links and extract relevant pages.
- `map`: Discover URLs from sitemaps, short crawls, and Common Crawl.
- `batch`: Process several URLs concurrently.
- `extract`: Convert page content into structured data.
- `summarize`: Produce a page summary.
- `diff`: Compare content snapshots.
- `brand`: Find colors, fonts, logos, and metadata.
- `search`: Search the web and scrape result pages.
- `research`: Run a multi-source research workflow.

## Limitations and notes

Crawling is restricted to same-origin links. Selector-based extraction depends on the target page's HTML structure, while pages requiring JavaScript are rendered before extraction. Windows source builds require platform tools including Rust, CMake, LLVM, NASM, Visual Studio Build Tools, and the Windows SDK; prebuilt binaries provide a simpler route. The Windows native installation guidance covers x64 binaries rather than ARM64.

_Full upstream README: https://allmcps.com/mcp/0xmassi-webclaw/readme_