Brightdata MCP vs PyreCrawl
A side-by-side look at the Brightdata MCP and PyreCrawl MCP servers: transport, auth, tools, quality signals, and the config to install either one in Claude, Cursor, Windsurf or VS Code.
Verdict: choose Brightdata MCP for search & data extraction work as a local process; choose PyreCrawl for search & data extraction as a local process. You can also run both side by side in one mcpServers block.
Which MCP server should you choose?
Choose Brightdata MCP when
- You need dedicated capabilities in the Search & Data Extraction domain.
- You prefer a local stdio subprocess transport.
- Its auth model fits you: API key required.
- You have the keys it needs:
API_TOKEN. - You want tools like
search_engine, scrape_as_markdown, search_engine_batch.
Choose PyreCrawl when
- You need dedicated capabilities in the Search & Data Extraction domain.
- You prefer a local stdio subprocess transport.
- Its auth model fits you: No auth required.
- You have the keys it needs:
PYRECRAWL_CACHE, PYRECRAWL_CACHE_TTL, PYRECRAWL_MONITOR_DIR, PYRECRAWL_NO_TELEMETRY, DO_NOT_TRACK. - You want tools like
scrape, extract, map_site.
Feature & specification comparison
Rows where the two servers differ are marked.
| Spec | ||
|---|---|---|
| Category | Search & Data Extraction | Search & Data Extraction |
| Quality signal | 73/100 · Great | 61/100 · Good |
| Install command | Confirmed | Confirmed |
| Transport | Local subprocess (stdio) | Local subprocess (stdio) |
| Auth | API key required | No auth required |
| Pricing | Freemium | Free |
| Required env vars | API_TOKEN | PYRECRAWL_CACHEPYRECRAWL_CACHE_TTLPYRECRAWL_MONITOR_DIRPYRECRAWL_NO_TELEMETRYDO_NOT_TRACK |
| Verified owner | Community listing | Community listing |
| GitHub stars | 2,686 | 22 |
| Directory activity | 5 views 0 installs 0 upvotes | 6 views 0 installs 0 upvotes |
| Tools | 11 | 13 |
| Full listing | View Brightdata MCP | View PyreCrawl |
Tools & capabilities
Brightdata MCP tools 11
search_engineScrape search results from Google, Bing or Yandex. Returns SERP results in JSON or Markdown (URL, title, description),Ideal forgathering current information, news, and detailed search results.scrape_as_markdownScrape a single webpage URL with advanced options for content extraction and get back the results in MarkDown language. This tool can unlock any webpage even if it uses bot detection or CAPTCHA.search_engine_batchRun multiple search queries simultaneously. Returns JSON for Google, Markdown for Bing/Yandex.scrape_batchScrape multiple webpages URLs with advanced options for content extraction and get back the results in MarkDown language. This tool can unlock any webpage even if it uses bot detection or CAPTCHA.discoverSearch the web and rank results by AI-driven relevance. Returns scored results with title, description, and URL. Supports intent-based ranking, geo-targeting, date filtering, and keyword filtering.search_scrapersFind a Bright Data pre-built scraper (Web Scraper API, 1,200+ sites) for a site or data type, e.g. "amazon reviews" or "linkedin.com". Returns dataset_id, name, domain and collection_methods. Next: call get_scraper_details with a dataset_id and one method.get_scraper_detailsShow the scraper description, input fields, example input and main output fields for one method. Call before run_scraper and build the input from sample_input or input_fields. Records may contain more fields than output_fields lists.run_scraperRun a scraper method. input is a list of objects shaped like sample_input from get_scraper_details, one per URL or query. collect_by_url returns one record per input; discover_by_* methods return up to limit_per_input records per input, and every record is billed. Waits up to 45 seconds; if still running it returns snapshot_id with status "running": then poll get_scraper_progress and call get_scraper_results when ready. Records with an "error" field explain failed inputs.get_scraper_progressStatus of a run: starting, running, ready, failed or canceled.get_scraper_resultsRecords of a finished run. Returns status "running" if not ready yet.refresh_scrapersReload the scraper list now instead of waiting for the daily refresh.
PyreCrawl tools 13
scrapeScrape a single URL → LLM-ready markdown. Use this when the user shares a URL and wants its content (read, analyze, summarize, extract). Auto-escalates through fast→stealth→llm when blocked. Args: url: Target URL (http/https). prefer: "auto" | "fast" | "stealth" | "llm". auto = fast first, escalate to stealth on block/short page. fast = cheap HTTP only (no JS). stealth = real Chromium + Cloudflare solver. llm = full Crawl4AI browser + BM25 fit-markdown. timeout: per-attempt timeout in seconds. include_html: include raw HTML in the response (large; off by default). js: (stealth only) JS expression evaluated against the live page after it settles. The value comes back in ``meta.js_result``. Use for data that lives in DOM *properties* (e.g. an input's ``.value``) rather than in serialized HTML. wait_for: (stealth only) JS predicate expression polled until truthy (bounded by ``timeout``). Use to wait for content that arrives asynchronously after ``network_idle``. Returns: {url, final_url, status, markdown, title, method, elapsed_ms, meta} or {error, url, method} on failure.extractScrape + structured extraction using a CSS-based JSON schema. Use when the user wants structured data (tables, lists, product info) extracted from a page. Define a CSS schema to target specific elements. The schema is a JsonCssExtractionStrategy schema: { "name": "PageItems", "baseSelector": "div.item", "fields": [{"name": "title", "selector": "h2", "type": "text"}, ...] } Returns parsed JSON in `data`.map_siteEnumerate all internal URLs reachable from `root`. Use when the user wants to map a site's structure or find all pages before crawling. Often paired with `crawl` or `batch_scrape`. Args: root: Website root (e.g. "https://example.com/docs"). include_pattern: Optional regex; only URLs matching are returned. limit: Hard cap on returned URLs.crawlMulti-page crawl: discover URLs on `root`, then scrape each. Use this when the user wants to crawl an entire site section or docs, or needs multiple pages scraped in bulk. For single pages use `scrape`; for research questions use `deep_research`. Args: root: start URL. max_pages: hard cap on pages scraped. css_selector: scope each page's html/markdown to the matched element (non-llm: lxml re-scope of the fetched HTML; llm: native crawl4ai css_selector). prefer: "auto" | "fast" | "stealth" | "llm" (llm = Crawl4AI BFS deep-crawl). include_paths: regex — keep only URLs matching (matched against full URL). exclude_paths: regex — drop URLs matching (e.g. `/tag/|/page/\d+`). max_depth: 0 = flat harvest from the root page's links (default); >0 = true BFS up to that link depth, honoring the filters. Returns: {root, pages: [{url, markdown, title, ...}], count, discovered, elapsed_ms} or {error, root} on failure.documentExtract text from a PDF/DOCX/PPTX URL → markdown (no browser). Use when the user shares a link to a document (PDF, Word, PowerPoint) and wants its text content. Also useful after `search_papers` to get full text from a paper's pdf_url. Content-type sniffed and routed to pypdf / python-docx / python-pptx. Optional deps — install with `pip install 'pyrecrawl[docs]'`.searchWeb search via DuckDuckGo HTML (no API key required). Use this for targeted searches where you need anti-bot bypass (Cloudflare protection on DDG). For simple searches, the built-in web_search may suffice. For research questions, prefer `deep_research` (search + scrape + citations). Returns [{url, title, snippet}, ...]. The smart ladder bypasses DDG's bot detection if needed. Returns: {query, results: [{url, title, snippet}], count} or {error, query} on failure.batch_scrapeScrape MANY URLs in ONE call (parallel, deduped, cache-aware). Use when the user provides multiple URLs or you have a list of pages to fetch. More efficient than calling `scrape` N times. Args: urls: Target URLs (deduped automatically; empties dropped). prefer: "auto" | "fast" | "stealth" | "llm". timeout: per-URL timeout in seconds. max_concurrency: parallel workers (default 4). include_html: include raw HTML per result (large; off by default). Returns {requested, unique, succeeded, failed, results[]}. Per-URL failures are isolated — other URLs still succeed. Returns: {requested, unique, succeeded, failed, results: [{url, markdown, ...}]}deep_researchSearch the web, then pull the top sources as EVIDENCE (no LLM synthesis). PRIMARY RESEARCH TOOL — use when the user asks to research, investigate, deep-dive, fact-check, or learn about a topic. Returns a ``citations`` list with stable [n] numbers and an ``evidence`` list of per-source markdown — the agent does the synthesis from evidence. Multi-pass mode: set iterations=2-3 to auto-run additional searches with refined queries (alternatives, criticism, latest developments) and append deduplicated evidence. Each pass adds up to ``scrape_top`` new sources. Args: query: search string. limit: how many search results to fetch. scrape_top: how many of those to actually fetch content from. prefer: "auto" | "fast" | "stealth" | "llm". iterations: 1 (default, single pass), 2-3 (multi-pass with refined queries targeting evidence gaps). Each pass searches from a different angle and deduplicates by URL. Returns: {query, iterations_run, queries: [str], hits: [{url, title, snippet}], citations: [{url, title}], evidence: [{url, title, markdown}], scraped, used_engines, elapsed_ms} or {error, query, hint} on failure.monitorTrack a URL over time and report meaningful content changes. Use when the user wants to watch a page for updates (price changes, new blog posts, status updates). Ask me to 'set up monitoring for <url>' for a guided setup playbook. Args: url: target URL. action: "check" | "history" | "forget". prefer: ladder preference, same as ``scrape``. css_selector: scope the diff to one element (so banner / nav changes don't trigger false positives). Snapshots persist under ``PYRECRAWL_MONITOR_DIR`` (default ``~/.pyrecrawl/monitors/``). ``check`` returns ``status`` of ``new`` | ``unchanged`` | ``changed`` | ``error`` and a unified diff when the page changed. Returns: {url, status: "new"|"unchanged"|"changed"|"error", diff?: str, snapshot_chars?: int, elapsed_ms?: int}search_papersSearch academic papers via arXiv or Crossref — no API keys. Args: query: free-text search, e.g. "transformer attention scaling laws". limit: max results (1-25 arXiv / 1-20 crossref). source: "arxiv" (CS/physics/math preprints, default) or "crossref" (all fields, DOI-backed). category: optional arXiv category filter, e.g. "cs.LG", "cs.CV". Returns papers with id/url/pdf_url/title/authors/summary/published. Feed pdf_url into the `document` tool to extract full text. Returns: {query, source, papers: [{title, authors, abstract, url, pdf_url?, ...}]} or {error, query} on failure.cacheInspect the response cache: stats, clear, enable, or disable. Use this to check cache hit rates before large batch jobs, or to clear stale cached responses when a site's content has changed. Args: action: "stats" (default) | "clear" | "disable" | "enable".healthVerify PyreCrawl is working: engine versions, dependencies, update status. Use this at the start of a session or before a large scraping job to confirm all engines are installed and up to date.- +1 more on the listing page
Ready-to-paste client configs
Paste either block (or both) into your client config, e.g. claude_desktop_config.json or ~/.cursor/mcp.json. We only show a config when the install command is confirmed.
Brightdata MCP
{
"mcpServers": {
"brightdata-mcp": {
"command": "npx",
"args": [
"@brightdata/mcp"
],
"env": {
"API_TOKEN": "YOUR_API_TOKEN_HERE"
}
}
}
}
PyreCrawl
{
"mcpServers": {
"pyrecrawl": {
"command": "uvx",
"args": [
"pyrecrawl@latest"
],
"env": {
"PYRECRAWL_CACHE": "YOUR_PYRECRAWL_CACHE_HERE",
"PYRECRAWL_CACHE_TTL": "YOUR_PYRECRAWL_CACHE_TTL_HERE",
"PYRECRAWL_MONITOR_DIR": "YOUR_PYRECRAWL_MONITOR_DIR_HERE",
"PYRECRAWL_NO_TELEMETRY": "YOUR_PYRECRAWL_NO_TELEMETRY_HERE",
"DO_NOT_TRACK": "YOUR_DO_NOT_TRACK_HERE"
}
}
}
}
Frequently asked questions
Brightdata MCP is categorized under Search & Data Extraction and uses a local stdio subprocess. PyreCrawl belongs to Search & Data Extraction and uses a local stdio subprocess. Pick Brightdata MCP for search & data extraction work and PyreCrawl for search & data extraction.
Copy the mcpServers block from the config section on this page into your client's config (for example claude_desktop_config.json or ~/.cursor/mcp.json), then fully restart the client.
Brightdata MCP is listed as freemium — a free tier with paid plans. Auth: api key required. PyreCrawl is listed as free. Auth: no auth required. If a server needs environment variables such as API keys, define them under the env key in your client's MCP config.
Yes. MCP clients support multiple servers: add brightdata-mcp and pyrecrawl as separate keys in the same mcpServers object, and your assistant will call tools from either one.
On AllMCPs, Brightdata MCP has a quality signal of 73/100 (Great) with 5 views, 0 installs and 2,686 GitHub stars. PyreCrawl has 61/100 (Good) with 6 views, 0 installs and 22 GitHub stars.