Log inSubmit server
Spectrawl logo

Spectrawl

by FayAndXan

Searches, browses, crawls, extracts, authenticates, and acts across web platforms through one self-hosted MCP package.

InstallRepositoryWebsite
Full README Compare alternatives Embed widget Claim this listing

Install Spectrawl

Pick your client, then add it in one click or paste the config.

STDIONode.js
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "fayandxan-spectrawl": {
      "command": "npx",
      "args": [
        "-y",
        "spectrawl"
      ]
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

We couldn’t confirm it starts correctly

We ran the install command but it didn't respond within our test window — this can mean a slow first-time install rather than a real problem.

npx -y spectrawl

No response to initialize.

This is an experimental automated check and can have false negatives — missing environment variables, a slow cold install, etc. It doesn’t necessarily mean something’s wrong. Last checked 1mo ago.

Overview

FayAndXan/spectrawl MCP server gives agents MCP tools for web search, stealth browsing, platform actions, authentication, and auth-status checks. It combines search-engine cascades, page scraping, browser automation, multi-page crawling, structured extraction, and platform adapters in a self-hosted Node.js package. Reach for it when an agent needs source content, browser interaction, stored login cookies, or actions on supported platforms rather than search snippets alone. Gemini Grounded Search is the default search path when a Gemini API key is configured, while keyless operation can fall back to DuckDuckGo.

Use cases

  • Search the web and return source pages for agent analysis
  • Browse blocked or JavaScript-heavy pages with stealth escalation
  • Crawl multiple pages and extract structured content
  • Reuse stored cookies for platform browsing and actions
  • Check authentication health across configured platforms

Key features

  • Multi-engine web search
  • Stealth URL browsing
  • Block-page detection
  • Multi-page crawling and extraction
  • Cookie-based authentication management
  • Platform action adapters

Tools5

What Spectrawl lets your agent do. Expand a tool for its parameters.

Self-reported~31 tokens of context

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked 2d ago

Recent health check succeeded.

22/22 checks healthy over the last 91d

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 4d ago via OSV.dev · spectrawl (npm)

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars28Stargazers on the source repository.
npm downloads34Package downloads in the last 30 days.
Last commit7mo agoMost recent push to the default branch.
Install checkInconclusiveDidn't respond in our test window — often a slow first install.
Tools exposed5Callable tools this server registers over MCP.
Directory activity4 viewsConfig copies, upvotes, and views on AllMCPs.

How Spectrawl works

What FayAndXan/spectrawl MCP server does

FayAndXan/spectrawl MCP server presents a unified web layer to an MCP client. Its exposed tools cover five areas: web_search for web queries, web_browse for retrieving pages, web_act for platform actions, web_auth for managing authentication, and web_status for checking authentication health.

The package is intended for agents that need more than a list of links. Search can return raw results or run a deeper pipeline that expands a query, searches in parallel, removes duplicates, ranks sources, scrapes selected pages, and optionally requests an AI-generated answer with citations. Browsing can return extracted Markdown and, when requested, raw HTML or a PNG screenshot.

How it works

The default search cascade uses Gemini Grounded Search, Tavily, and Brave. DuckDuckGo, Bing, Serper, Google CSE, Jina Reader, and SearXNG are also listed as available engines. Gemini Grounded Search has a stated free allowance of 5,000 grounded queries per month, while other engines have their own key or usage requirements.

For page access, the browser path starts with Playwright and a stealth plugin. If that path is blocked, the system can escalate to an installed Camoufox binary or a configured remote Camoufox deployment. Responses report the selected engine and can identify common block or challenge pages, including Cloudflare, Akamai, AWS WAF, Imperva, DataDome, PerimeterX/HUMAN, hCaptcha, and reCAPTCHA.

The package also supports multi-page crawling, structured extraction, network capture, an AI browser agent, and 24 platform adapters. Stored cookies can be supplied for supported platform browsing and actions. Some sites use alternative paths automatically: Reddit can use PullPush, Amazon can use Jina Reader, and X/Twitter can use the xAI Responses API when configured.

Setup and configuration

Install the Node.js package with npm install spectrawl. Without any search keys, the package can use DuckDuckGo for raw, keyless search, although datacenter IPs may be rate-limited.

Additional providers use TAVILY_API_KEY, BRAVE_API_KEY, SERPER_API_KEY, GOOGLE_CSE_KEY, or XAI_API_KEY according to the selected engine or fallback. Camoufox can be installed with npx spectrawl install-stealth. LinkedIn access requires a residential proxy and stored LinkedIn cookies; the README gives CLI examples for configuring a proxy and importing cookies.

Tools and capabilities

  • Search the web with basic or deep search modes.
  • Browse URLs with stealth options, screenshots, HTML output, caching controls, and stored authentication.
  • Detect block pages and report block details instead of returning unusable content.
  • Crawl multiple pages and extract structured information.
  • Perform platform actions through the available adapters.
  • Manage cookies and inspect authentication health.
  • Use Gemini Vision for supported CAPTCHA-solving workflows.

Limitations and notes

FayAndXan/spectrawl MCP server does not guarantee access to sites that reject datacenter traffic. LinkedIn is specifically described as requiring a residential proxy because it associates cookies with the originating IP. X/Twitter fallback access requires XAI_API_KEY, and some alternatives have their own costs or usage limits.

Gemini Grounded Search, CAPTCHA solving, and third-party search or fallback services can consume provider quotas.

Read the full README View source on GitHub

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about Spectrawl

The README instructs users to install the package with `npm install spectrawl`. It does not provide a separate MCP server launch command.

Gemini uses `GEMINI_API_KEY`; the documented optional provider variables include `TAVILY_API_KEY`, `BRAVE_API_KEY`, `SERPER_API_KEY`, `GOOGLE_CSE_KEY`, and `XAI_API_KEY`.

Yes. Datacenter IPs may be rate-limited.

It can escalate from Playwright with stealth to Camoufox and reports detected block pages. Site-specific fallbacks are also documented for Reddit, Amazon, and X/Twitter.

Related MCP servers

All in Search & Data Extraction All alternatives
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewSpectrawl AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/fayandxan-spectrawl?style=directory)](https://allmcps.com/mcp/fayandxan-spectrawl)
HTML Embed
<a href="https://allmcps.com/mcp/fayandxan-spectrawl"><img src="https://allmcps.com/api/badge/fayandxan-spectrawl?style=directory" alt="Spectrawl on AllMCPs" /></a>