Log inSubmit server
Webscraping AI MCP Server logo

Webscraping AI MCP Server

by webscraping-ai

Extracts web page content, structured fields, HTML, CSS-selected data, and Google results through WebScraping.AI.

InstallRepositoryWebsite
Full README Compare alternatives Embed widget Claim this listing

Install Webscraping AI MCP Server

Pick your client, then add it in one click or paste the config.

STDIONode.js
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "webscraping-ai-webscraping-ai-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "webscraping-ai-mcp"
      ],
      "env": {
        "WEBSCRAPING_AI_API_KEY": "YOUR_VALUE_HERE",
        "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "YOUR_VALUE_HERE",
        "WEBSCRAPING_AI_DEFAULT_PROXY_TYPE": "YOUR_VALUE_HERE",
        "WEBSCRAPING_AI_DEFAULT_JS_RENDERING": "YOUR_VALUE_HERE",
        "WEBSCRAPING_AI_DEFAULT_TIMEOUT": "YOUR_VALUE_HERE",
        "WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT": "YOUR_VALUE_HERE",
        "WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING": "YOUR_VALUE_HERE"
      }
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

Values are written into the snippet above so you can paste it as is. They never leave your browser.

Install not yet sandbox-tested— we haven't run this command through our automated sandbox yet. Not a red flag; we're working through the catalog.

Overview

webscraping-ai/webscraping-ai-mcp-server connects MCP clients to WebScraping.AI for browser-rendered web extraction, Google search results, and structured answers from public pages. Its tools can retrieve HTML or visible text, select content with CSS selectors, extract instructed fields, and ask questions about a page. Google searches support location, language, country, pagination, and up to five consecutive pages. Reach for it when an agent needs rendered web content, proxy selection, custom JavaScript, or parsed search data rather than direct HTTP fetching.

Use cases

  • Extract product fields from rendered pages
  • Ask questions about public web pages
  • Retrieve HTML or visible text for analysis
  • Collect CSS-selected page content
  • Search Google from a specified location

Key features

  • JavaScript-rendered HTML and text extraction
  • Instruction-based structured field extraction
  • CSS selector extraction for one or multiple elements
  • Location-aware Google SERP retrieval across up to five pages
  • Datacenter, residential, and stealth proxy selection
  • Optional scraped-content sandboxing

Tools11

What Webscraping AI MCP Server lets your agent do. Expand a tool for its parameters.

Self-reported~370 tokens of context

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked 2d ago

Recent health check succeeded.

17/18 checks healthy over the last 80d

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 10d ago via OSV.dev · webscraping-ai-mcp (npm)

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars45Stargazers on the source repository.
npm downloads1.1kPackage downloads in the last 30 days.
Last commit3d agoMost recent push to the default branch.
Tools exposed11Callable tools this server registers over MCP.
Directory activity5 viewsConfig copies, upvotes, and views on AllMCPs.

How Webscraping AI MCP Server works

What webscraping-ai/webscraping-ai-mcp-server does

WebScraping.AI exposes web data extraction functions through MCP. The server connects an MCP client to the WebScraping.AI API, which can retrieve public pages with headless Chrome or Chromium, apply proxy settings, and process page content.

Available operations include asking a question about a page, extracting named fields into structured data, retrieving rendered HTML, obtaining visible text, and selecting one or multiple elements with CSS selectors. It also provides parsed Google search results, structured JSON extraction from a page URL, and account usage monitoring. Content sandboxing can wrap retrieved pages and search results in boundaries that identify them as external content.

How it works

The local package runs as an MCP stdio process. An MCP client starts the process and supplies the WEBSCRAPING_AI_API_KEY environment variable. Requests are sent to WebScraping.AI, where JavaScript rendering, proxy selection, timeouts, device emulation, and custom page scripts can be applied according to the selected operation and configuration.

Google search requests use the google engine by default. They can specify language, country, location, Google-encoded location, proxy country, radius, page number, and a page count from one to five. When multiple pages are fetched, duplicate results are removed and each result includes its source page.

Setup and configuration

Run the package directly with npx:

bash
env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp

For a local checkout, clone the repository, run npm install, and start it with npm start. A WebScraping.AI API key is required for every operation; the service offers a free trial without a credit card.

Optional environment variables control request behavior:

  • WEBSCRAPING_AI_CONCURRENCY_LIMIT sets the maximum number of simultaneous requests and defaults to 5.
  • WEBSCRAPING_AI_DEFAULT_PROXY_TYPE selects datacenter, residential, or stealth proxies; the default is residential.
  • WEBSCRAPING_AI_DEFAULT_JS_RENDERING controls JavaScript rendering, enabled by default.
  • WEBSCRAPING_AI_DEFAULT_TIMEOUT sets page retrieval time in milliseconds, with a maximum of 30,000.
  • WEBSCRAPING_AI_DEFAULT_JS_TIMEOUT sets the JavaScript rendering limit.
  • WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING enables security boundaries around scraped content and search results.

Cursor can load the server from a project or global mcp.json file, and Claude Desktop can start it through claude_desktop_config.json. Cursor configuration requires version 0.45.6 or newer.

Tools and capabilities

The extraction tools cover several output styles:

  • Ask natural-language questions about page content.
  • Map extraction instructions to fields such as product title, price, and description.
  • Return full rendered HTML or visible page text.
  • Extract one CSS-selected element or multiple selector matches.
  • Run Google searches with location-aware results and up to five pages per call.
  • Use country-specific proxies, JavaScript rendering, custom JavaScript, and desktop, mobile, or tablet emulation.

Search requests accept additional site-specific query parameters through params. Keys must use letters, numbers, underscores, or hyphens and cannot override reserved request fields such as url or api_key.

Read the full README View source on GitHub

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about Webscraping AI MCP Server

webscraping-ai/webscraping-ai-mcp-server is an MCP server that connects MCP clients to WebScraping.AI for web extraction. Its main tools ask questions about pages, extract structured fields, return HTML or visible text, select CSS-matched content, and retrieve parsed Google search results with location and pagination controls.

Install it by running `npx -y webscraping-ai-mcp` with `WEBSCRAPING_AI_API_KEY` set, or clone the repository, run `npm install`, and start it with `npm start`. The API key is required for all operations.

Yes, it works with Cursor through a project or global `.cursor/mcp.json` file. Configure the server as a command using `npx -y webscraping-ai-mcp` and provide the API key in the `env` object. Cursor version 0.45.6 or newer is required.

Set `WEBSCRAPING_AI_DEFAULT_JS_RENDERING` for the default rendering behavior and `WEBSCRAPING_AI_DEFAULT_PROXY_TYPE` to `datacenter`, `residential`, or `stealth`. Individual requests can also specify JavaScript, timeouts, proxy type, country, and a wait-for selector.

Yes, set `WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING=true` to wrap scraped content and search results with boundaries identifying them as external content. This helps an LLM distinguish retrieved text from system instructions.

Related MCP servers

All in Search & Data Extraction All alternatives
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewWebscraping AI MCP Server AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/webscraping-ai-webscraping-ai-mcp-server?style=directory)](https://allmcps.com/mcp/webscraping-ai-webscraping-ai-mcp-server)
HTML Embed
<a href="https://allmcps.com/mcp/webscraping-ai-webscraping-ai-mcp-server"><img src="https://allmcps.com/api/badge/webscraping-ai-webscraping-ai-mcp-server?style=directory" alt="Webscraping AI MCP Server on AllMCPs" /></a>