Spectrawl

What FayAndXan/spectrawl MCP server does

FayAndXan/spectrawl MCP server presents a unified web layer to an MCP client. Its exposed tools cover five areas: web_search for web queries, web_browse for retrieving pages, web_act for platform actions, web_auth for managing authentication, and web_status for checking authentication health.

The package is intended for agents that need more than a list of links. Search can return raw results or run a deeper pipeline that expands a query, searches in parallel, removes duplicates, ranks sources, scrapes selected pages, and optionally requests an AI-generated answer with citations. Browsing can return extracted Markdown and, when requested, raw HTML or a PNG screenshot.

How it works

The default search cascade uses Gemini Grounded Search, Tavily, and Brave. DuckDuckGo, Bing, Serper, Google CSE, Jina Reader, and SearXNG are also listed as available engines. Gemini Grounded Search has a stated free allowance of 5,000 grounded queries per month, while other engines have their own key or usage requirements.

For page access, the browser path starts with Playwright and a stealth plugin. If that path is blocked, the system can escalate to an installed Camoufox binary or a configured remote Camoufox deployment. Responses report the selected engine and can identify common block or challenge pages, including Cloudflare, Akamai, AWS WAF, Imperva, DataDome, PerimeterX/HUMAN, hCaptcha, and reCAPTCHA.

The package also supports multi-page crawling, structured extraction, network capture, an AI browser agent, and 24 platform adapters. Stored cookies can be supplied for supported platform browsing and actions. Some sites use alternative paths automatically: Reddit can use PullPush, Amazon can use Jina Reader, and X/Twitter can use the xAI Responses API when configured.

Setup and configuration

Install the Node.js package with npm install spectrawl. The README shows GEMINI_API_KEY as the key used for Gemini search and recommends obtaining it from Google AI Studio. Without any search keys, the package can use DuckDuckGo for raw, keyless search, although datacenter IPs may be rate-limited.

Additional providers use TAVILY_API_KEY, BRAVE_API_KEY, SERPER_API_KEY, GOOGLE_CSE_KEY, or XAI_API_KEY according to the selected engine or fallback. Camoufox can be installed with npx spectrawl install-stealth. LinkedIn access requires a residential proxy and stored LinkedIn cookies; the README gives CLI examples for configuring a proxy and importing cookies.

Tools and capabilities

• Search the web with basic or deep search modes.
• Browse URLs with stealth options, screenshots, HTML output, caching controls, and stored authentication.
• Detect block pages and report block details instead of returning unusable content.
• Crawl multiple pages and extract structured information.
• Perform platform actions through the available adapters.
• Manage cookies and inspect authentication health.
• Use Gemini Vision for supported CAPTCHA-solving workflows.

Limitations and notes

FayAndXan/spectrawl MCP server does not guarantee access to sites that reject datacenter traffic. LinkedIn is specifically described as requiring a residential proxy because it associates cookies with the originating IP. X/Twitter fallback access requires XAI_API_KEY, and some alternatives have their own costs or usage limits.

Deep search does not create an AI summary unless summarization is explicitly enabled; by default it returns source material for the agent's own model to process. Gemini Grounded Search, CAPTCHA solving, and third-party search or fallback services can consume provider quotas. The README does not identify a software license or list specific MCP client compatibility.