ScraperAPI MCP server

The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.

This is the self-hosted (local) server. A hosted (remote) version is also available.

Table of Contents

• Features
• Architecture
• Installation
• Available tools
• Configuration
  • Settings
  • Client Setup
• Development

Features

• Full implementation of the Model Context Protocol specification
• Seamless integration with ScraperAPI for web scraping
• Simple setup with Python or Docker

Architecture

          ┌───────────────┐     ┌───────────────────────┐     ┌───────────────┐
          │  LLM Client   │────▶│  Scraper MCP Server   │────▶│    AI Model   │
          └───────────────┘     └───────────────────────┘     └───────────────┘
                                            │
                                            ▼
                                  ┌──────────────────┐
                                  │  ScraperAPI API  │
                                  └──────────────────┘

Installation

The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.

Prerequisites

• Python 3.11+
• Docker (optional)

Using Python

Install the package:

pip install scraperapi-mcp-server

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "python",
      "args": ["-m", "scraperapi_mcp_server"],
      "env": {
        "API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
      }
    }
  }
}

Using Docker

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "-e",
        "API_KEY=${API_KEY}",
        "--rm",
        "scraperapi-mcp-server"]
    }
  }
}

[!TIP]

If your command is not working (for example, you see a package not found error when trying to start the server), double-check the path you are using. To find the correct path, activate your virtual environment first, then run:

bash

Copy

which <YOUR_COMMAND>

Available tools

API

SDEs

Structured Data Endpoints (SDEs) return pre-parsed JSON (or CSV) instead of raw HTML. Every SDE tool also accepts output_format (json by default, or csv), tld, and country_code, shown in each tool's table.

Google

Amazon

Walmart

eBay

Redfin

Every Redfin tool takes a full Redfin URL matching the tool (property, search, or agent page).

Crawler

The crawler is asynchronous: crawler_job_start returns a job id immediately, then you poll crawler_job_status until the crawl finishes. Per-page results can also be pushed to a callback_url webhook.

AI Parser

Build a reusable parser from a few example URLs, then apply it to any similar page for structured extraction. Two-phase: ai_parser_create returns a parser id immediately and generation runs in the background — poll ai_parser_get_details until its status is FINISHED, then call ai_parser_parse_url.

Prompt templates

• Please scrape this URL <URL>. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.
• Can you scrape URL <URL> to extract <SPECIFIC_DATA>? If the request returns missing/incomplete <SPECIFIC_DATA>, set render=true to enable JS Rendering.

Configuration

Settings

Configure the server through environment variables. Only API_KEY is required.

Client Setup

Use the JSON configuration file from the Installation section. Below are steps for common clients.

Development

Local setup

1. Clone the repository:

   bash

   Copy

   git clone https://github.com/scraperapi/scraperapi-mcp
   cd scraperapi-mcp
   

2. Install dependencies:

   • Using Poetry:
     bash

     Copy

     poetry install
     

   • Using pip:
     bash

     Copy

     # Create virtual environment and activate it
     python -m venv .venv
     source .venv/bin/activate # MacOS/Linux
     # OR
     .venv/Scripts/activate # Windows
     
     # Install the local package in editable mode
     pip install -e .
     

   • Using Docker:
     bash

     Copy

     # Build the Docker image locally
     docker build -t scraperapi-mcp-server .
     

Run the server

• Using Python:
  bash

  Copy

  python -m scraperapi_mcp_server
  

• Using Docker:
  bash

  Copy

  # Run the Docker container with your API key
  docker run -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> scraperapi-mcp-server
  

Debug

python3 -m scraperapi_mcp_server --debug

Testing

This project uses pytest for testing.

Install Test Dependencies

• Using Poetry:
  bash

  Copy

  poetry install --with dev
  

• Using pip:
  Terminal

  Copy

  pip install -e .
  pip install pytest pytest-mock pytest-asyncio
  

Running Tests

# Run All Tests
pytest

# Run Specific Test
pytest <TEST_FILE_PATH>