Webscraping AI MCP Server

What webscraping-ai/webscraping-ai-mcp-server MCP server does

The webscraping-ai/webscraping-ai-mcp-server MCP server connects an MCP client to WebScraping.AI’s web extraction API. It is intended for agents that need to retrieve or interpret content from public web pages without implementing browser automation and proxy handling themselves.

Supported tasks include asking a question about a page, extracting named fields from a page, retrieving rendered HTML, reading visible text, and selecting one or more elements with CSS selectors. The service can render JavaScript with headless Chrome or Chromium, which makes it applicable to pages whose useful content is loaded after the initial response.

How it works

The local server communicates with an MCP client through stdio. Each tool receives a target URL and task-specific arguments. Requests can control JavaScript rendering, rendering and retrieval timeouts, a selector to wait for, proxy type, and country. Proxy options include datacenter, residential, and stealth proxies, while device emulation supports desktop, mobile, and tablet contexts according to the project description.

The server sends extraction requests to WebScraping.AI and returns the resulting text, HTML, selected content, or structured response to the client. It also supports custom JavaScript execution, concurrent-request limits, and account usage monitoring. Content sandboxing can wrap returned material with boundaries identifying it as external content, helping an agent distinguish scraped text from instructions.

Setup and configuration

Install the npm package with npx -y webscraping-ai-mcp, then provide WEBSCRAPING_AI_API_KEY. The key is required for all operations and is obtained from WebScraping.AI. The repository also documents cloning the source, running npm install, and starting the project with npm start for a manual setup.

Optional settings control the maximum concurrent requests, default proxy type, whether JavaScript rendering is enabled by default, the page retrieval timeout, and the JavaScript timeout. Content sandboxing is disabled by default and can be enabled with WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING=true.

The README provides configuration examples for Cursor and Claude Desktop. The hosted remote endpoint is a separate zero-local-install option that uses WebScraping.AI account sign-in and OAuth; the open-source package described here is the stdio version and uses an API key.

Tools and capabilities

The webscraping-ai/webscraping-ai-mcp-server MCP server documents these tools:

• webscraping_ai_question asks a question about page content.
• webscraping_ai_fields extracts fields described by user-provided instructions.
• webscraping_ai_html returns a page’s HTML, with optional JavaScript rendering.
• webscraping_ai_text returns visible page text.
• webscraping_ai_selected extracts content matching one CSS selector.
• webscraping_ai_selected_multiple extracts content matching multiple selectors.

Tool arguments can override defaults for options such as JavaScript, timeouts, waiting for a page selector, proxy type, and country. The fields tool is suitable for results such as product titles, prices, and descriptions when those values are present on the target page.

Limitations and notes

Use of the local webscraping-ai/webscraping-ai-mcp-server MCP server depends on WebScraping.AI availability and account credits. The README states that a free trial includes 2,000 credits and does not require a credit card, but it does not define pricing after the trial.

The default retrieval timeout is 15,000 milliseconds, with a documented maximum of 30,000 milliseconds; the default JavaScript timeout is 2,000 milliseconds. These values can affect slow or heavily client-rendered pages. Content sandboxing marks scraped content as external but does not turn that content into a trusted source or guarantee protection from every prompt-injection technique.