# Spicrawl [Health: Active]

**Category:** 🔎 Search & Data Extraction  
**Repository:** https://github.com/OfficialSpicrawl/mcp  
**GitHub Stars:** 0  
**Views:** 0  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/spicrawl

## Description
Web scraping to Markdown and structured data, screenshots, batch jobs and browser sessions.

## Claude Desktop Quick Installation
Install path detected from listing signals. Uses `npx` (confidence: high):

```json
"mcpServers": {
  "spicrawl": {
    "command": "npx",
    "args": ["-y","@spicrawl/mcp"],
    "env": {
      "SPICRAWL_API_KEY": ""
    }
  }
}
```

**Requires environment variables:** `SPICRAWL_API_KEY` — the values above are empty placeholders; fill in real credentials before running (see the repository for what each one is for).

## Documentation

## What Spicrawl MCP server does

The Spicrawl MCP server exposes web scraping and browser-assisted extraction tools to MCP-compatible agents. Its main single-page tool, `spicrawl_scrape`, can return a page as Markdown, plain text, HTML, a JSON envelope, or a printed PDF. Markdown is the default and focuses on the page's main content.

For the Search & Data Extraction category, the server is suited to agents that need current content from websites rather than a static local dataset. It can fetch ordinary pages, render JavaScript-driven pages, extract fields with CSS or XPath selectors, and read metadata such as JSON-LD, OpenGraph, Twitter Card, and microdata through automatic parsing.

The service also supports screenshots, typed browser actions, asynchronous batch jobs, persistent browser sessions, usage records, request logs, and documentation lookup. Check a target site's terms and `robots.txt` before collecting content.

## How it works

Spicrawl MCP server calls can use either the hosted endpoint at `https://mcp.spicrawl.com/mcp` or the npm package `@spicrawl/mcp`. The hosted option uses MCP Streamable HTTP and does not require a local installation. The local option uses Node.js 20 or newer and communicates over stdio. Both transports expose the same tool family and apply the API key's scopes, limits, and credit accounting.

A basic scrape can use a normal fetch. Set `render: true` when a page needs JavaScript execution, or use browser actions such as `click`, `fill`, `wait_for`, `wait_for_navigation`, `scroll`, `select`, `evaluate`, and `screenshot` before capturing the result. The `extract` option accepts selector mappings, while `autoparse` reads structured metadata already published by the page.

For larger collections, submit an asynchronous batch job and then inspect its status and results. Jobs can contain up to 10,000 URLs. Open jobs can accept additional URLs before they are closed. A persistent session can retain cookies and browser storage for later requests, including authenticated workflows.

## Setup and configuration

Create an API key through the Spicrawl account site. Keys use the `spicrawl_live_` or `spicrawl_test_` prefixes. Set it in `SPICRAWL_API_KEY`, then configure the hosted endpoint with an `Authorization: Bearer` header, or pass the variable to the local `npx -y @spicrawl/mcp` process.

The server is documented for Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI, and Codex. The README also provides direct installation paths through npm, Smithery, Glama, the MCP Registry, Dify, and Cursor Directory.

## Tools and capabilities

Spicrawl MCP server includes tools for:

- Scraping one URL into Markdown, text, HTML, JSON, or PDF.
- Rendering JavaScript pages and performing browser actions.
- Extracting selected fields or published page metadata.
- Returning screenshots as MCP image blocks and PDFs as resource blocks.
- Submitting, monitoring, paging, retrying, cancelling, extending, and closing batch jobs.
- Creating persistent browser sessions for logged-in pages.
- Reviewing request details, usage, and errors.
- Searching Spicrawl documentation from within an agent.

The package defines 25 tools. The hosted endpoint currently lists 24 because `spicrawl_browser_connect_url` is not included there until remote browser support launches.

## Limitations and notes

Credit usage depends on the operation. A plain fetch costs 1 credit, JavaScript rendering costs 3, and screenshots, PDFs, or browser actions use the full browser tier at 8 credits. Automatic mode can select the least expensive tier that completes the request. The free tier includes 1,000 credits per month.

Screenshots are limited to 5 MB per base64 image block, and PDF resources are limited to 10 MB. Tool arguments are validated, so unknown or misspelled fields return errors. Scraping, chunking, embedding, and scheduling downstream workflows remain the responsibility of the consuming application.

_Full upstream README: https://allmcps.com/mcp/spicrawl/readme_