WET - Web Extended Toolkit MCP Server

mcp-name: io.github.n24q02m/wet-mcp

Open-source MCP server for AI agents: web search, content extraction, and library docs.

Current release: v3.x. media(action="analyze") was removed in the v2.0.0 BREAKING release. Use imagine-mcp's understand action for vision/audio/video analysis. See docs/migration.md for the upgrade recipe.

Table of contents

• Features
• Status
• Quick install
• Configuration
• Documentation
• Tools
• CLI
• Comparison
• Security
• Build from Source
• Deploy to Cloudflare
• Smithery
• Trust Model
• License

Features

• Web Search -- Embedded SearXNG metasearch (Google, Bing, DuckDuckGo, Brave) with query expansion, TTL cache (1 h general / 5 min time-sensitive), standardized citation format, and 200-token snippet cap. Optional cloud search backends (Tavily, Brave, Exa) as a fallback chain via SEARCH_BACKENDS
• Academic Research -- Search Google Scholar, Semantic Scholar, arXiv, PubMed, CrossRef, BASE
• Library Docs -- Auto-discover and index documentation with FTS5 hybrid search, HyDE-enhanced retrieval, and version-specific docs
• Content Extract -- 5-strategy escalation chain via n24q02m-web-core ScrapingAgent (basic_http -> tls_spoof -> render backends from BROWSER_BACKENDS (native / browserless / cf-browser-rendering) -> optional key-gated captcha), markitdown bridge for low-tier HTML/MD fallback, smart chunks structured output (clean text + markdown + JSON-LD + code blocks + metadata), batch processing (up to 50 URLs), deep crawling, site mapping
• Local File Conversion -- Convert PDF, DOCX, XLSX, CSV, HTML, EPUB, PPTX to Markdown
• Media -- List + download images / videos / audio files. analyze was removed in v2.0.0 -- use imagine-mcp.understand for vision/audio inference
• Anti-bot -- Stealth strategies bypass Cloudflare, Medium, LinkedIn, Twitter
• Zero Config -- Built-in local reference embedding + reranking through fastretrieval, no API keys needed. Optional cloud providers (Jina AI, Gemini, OpenAI, Cohere, xAI, Anthropic) selected per task via the EMBEDDING_MODELS / RERANK_MODELS / LLM_MODELS model chains for higher-quality vectors and LLM features
• Sync -- Cross-machine sync of indexed docs via Google Drive (OAuth Device Code, no browser redirect)

Quick install

# Method 1 (default): plugin install via Claude Code
/plugin marketplace add n24q02m/claude-plugins
/plugin install wet-mcp@n24q02m-plugins

# Method 2 (CLI): direct uvx invocation
claude mcp add wet -- uvx wet-mcp

# Method 3 (source-built container for HTTP / multi-device / OAuth)
docker build --target http -t wet-mcp:local .
docker run -d --name wet-mcp-http -p 8084:8080 \
  -v wet-data:/data -e PUBLIC_URL=https://wet.example.com \
  wet-mcp:local

# Method 4 (remote): point a client at an HTTP deployment
claude mcp add --transport http wet https://<your-host>/mcp

Public OCI image publication is discontinued. Existing historical registry tags remain untouched; new container deployments build from source or use the Cloudflare-managed registry.

The HTTP endpoint speaks Streamable HTTP and is OAuth-gated -- your client is prompted to authenticate in the browser on first connect (no API key to paste). Stand one up via Method 3 or the Deploy to Cloudflare section.

Full setup matrices live at the canonical docs site mcp.n24q02m.com/servers/wet-mcp/setup/ and the paste-to-agent snippets at claude-plugins/plugins/wet-mcp/setup-with-agent.md (per Spec F single source of truth).

Configuration

wet runs zero-config out of the box: web search uses an embedded local SearXNG, and embedding/reranking fall back to the bundled local ONNX models through fastretrieval when no cloud keys are set. For higher-quality results, point each task at a cloud model chain. All settings are plain environment variables (no app prefix) -- in the HTTP self-host mode they are entered through the browser setup form instead.

Model chains (CSV provider/model,provider/model; order = fallback). Leave a chain empty to use the local ONNX models (embedding/rerank) or to disable LLM features (LLM):

Provider keys -- the provider is inferred from each model's prefix; supply the matching key (litellm <PROVIDER>_API_KEY convention):

Any other litellm provider works via env passthrough -- see litellm provider docs for its key name.

FASTRETRIEVAL_CACHE_PATH controls the local model cache.

Search backends -- SEARCH_BACKENDS is an ordered runtime fallback chain: searxng (default, local or external via SEARXNG_URL), keyed tavily / brave / exa / kagi, optional-key firecrawl, and credential-free duckduckgo / startpage. Keyed providers use TAVILY_API_KEY, BRAVE_API_KEY, EXA_API_KEY, or KAGI_API_KEY. Firecrawl attempts a keyless request when FIRECRAWL_API_KEY is absent; rejection or a DuckDuckGo/Startpage bot challenge advances the chain. The same chain serves web search, research, similar-page search, agent search, and docs discovery/indexing fallbacks. SearXNG retains its science-category filter for research; other providers use their own search capabilities. Hosted users configure the chain and keys in their own relay record. An empty hosted record never inherits an operator's provider key or local SearXNG URL; single-user stdio still uses env/settings and preserves the public local path.

Browser render backends -- BROWSER_BACKENDS (CSV, escalation chain) picks the headless render leg of extract: native (in-process chromium, the zero-config default), browserless (self-host render service -- set BROWSERLESS_URL + BROWSERLESS_TOKEN), and cf-browser-rendering (Cloudflare Browser Rendering -- set CF_ACCOUNT_ID + CF_BROWSER_RENDERING_TOKEN). Empty chain falls back to native. Set CAPSOLVER_API_KEY to append an optional, key-gated CAPTCHA tier as the last escalation step.

Robots policy -- set RESPECT_ROBOTS_TXT=true to enforce robots.txt across both the extract strategy chain and the Crawl4AI-backed crawl, sitemap, and list_media actions. The default is false to preserve existing deployment behaviour; configure this process-level policy explicitly when the operator requires robots enforcement.

Disable local fallbacks -- opt out of the heavy in-process local fallbacks per capability (e.g. on a slim container that renders/searches/embeds via cloud backends only): DISABLE_LOCAL_BROWSER, DISABLE_LOCAL_SEARCH, DISABLE_LOCAL_EMBED, DISABLE_LOCAL_RERANK.

Docs sync -- SYNC_ENABLED (default true), GOOGLE_DRIVE_CLIENT_ID (required for sync), SYNC_FOLDER (default wet-mcp), SYNC_INTERVAL (default 300s). Sync uses Google Drive over the OAuth Device Code flow (no browser redirect). DOCS_DB_BACKEND=cf-d1 disables GDrive/S3 file sync, including automatic startup, relay wizard and device-code setup, even if legacy sync settings remain. Non-CF SQLite deployments retain GDrive sync; SYNC_S3_BUCKET selects S3 instead, and SYNC_ENABLED=false disables both.

HTTP self-host -- MCP_TRANSPORT=http, PUBLIC_URL=<your-domain>. The setup form is gated by MCP_RELAY_PASSWORD; multi-user deployments require CREDENTIAL_SECRET (per-user vault key), MCP_JWT_SIGNING_SECRET (rotatable OAuth JWT key), and MCP_DCR_SERVER_SECRET.

Example stdio config (cloud chains):

{
  "mcpServers": {
    "wet": {
      "command": "uvx",
      "args": ["wet-mcp"],
      "env": {
        "EMBEDDING_MODELS": "jina_ai/jina-embeddings-v5-text-small",
        "RERANK_MODELS": "jina_ai/jina-reranker-v3",
        "LLM_MODELS": "gemini/gemini-3-flash-preview",
        "JINA_AI_API_KEY": "jina_xxx",
        "GEMINI_API_KEY": "AIza_xxx"
      }
    }
  }
}

Status

Stable architecture with two transports: stdio (default, local) and HTTP (self-host, OAuth-gated). No daemon-bridge layer and no auto-spawn from stdio. The media.analyze action was removed in the v2.0.0 BREAKING release -- see docs/migration.md for the upgrade recipe. Current release line: v3.x.

Documentation

Full docs at mcp.n24q02m.com/servers/wet-mcp/setup/:

• Setup -- install methods for Claude Code, Codex, Gemini CLI, Cursor, Windsurf, mcp.json
• Modes overview -- stdio / local-relay / remote-relay / remote-oauth
• Multi-user setup -- per-JWT-sub credential model

In-repo references (Spec F single source of truth: setup docs live in claude-plugins/plugins/wet-mcp/):

• docs/ARCHITECTURE.md -- web-core ScrapingAgent integration, strategy chain, storage layout, LLM provider dispatch
• docs/BENCHMARKS.md -- v1.x baseline coverage / latency placeholders + tier-1 fixture metrics

Install with AI agent -- paste this to your AI coding agent:

Install MCP server wet-mcp following the steps at https://raw.githubusercontent.com/n24q02m/claude-plugins/main/plugins/wet-mcp/setup-with-agent.md

Tools

6 MCP tools (3 domain + config + help + config__open_relay). The legacy setup tool merged into config action dispatch.

Media boundary: For vision / audio understanding (image captioning, OCR, audio transcription, video summarization), use imagine-mcp. media.analyze was removed in wet v2.0.0 -- use imagine-mcp.understand instead.

CLI

The wet-mcp console script starts the server and also exposes a few one-shot operator subcommands. A bare invocation (or any leading-dash flag) starts the server; a leading positional argument is dispatched as a subcommand.

wet-mcp                        # start the server over stdio (default transport)
wet-mcp --http                 # start the server over Streamable HTTP (self-host mode)

wet-mcp auth google            # authorize the Google credential provider for Drive sync
wet-mcp logout                 # clear the local Google Drive sync token
wet-mcp warmup                 # pre-download local models + run auto-setup (SearXNG, browser) to avoid first-run delays
wet-mcp docs reindex <library> # drop the cached docs index for <library>; the next docs search re-indexes it

auth google accepts an optional bring-your-own OAuth client via --client-id and --client-secret (single-user / local machine only; the token is written to the local store). Each subcommand prints a JSON result and exits.

Security

• SSRF prevention -- URL validation on crawl targets
• Graceful fallbacks -- Cloud → Local embedding, multi-tier crawling
• Error sanitization -- No credentials in error messages
• File conversion sandboxing -- Optional CONVERT_ALLOWED_DIRS restriction

Build from Source

git clone https://github.com/n24q02m/wet-mcp.git
cd wet-mcp
uv sync
uv run wet-mcp

Deploy to Cloudflare

Run your own single-user wet instance serverless on Cloudflare (Containers + D1 + Vectorize + KV).

Prerequisites: a Cloudflare account on the Workers Paid plan — required for Containers, D1, and Vectorize (the Cloudflare free tier does not include them) — and the wrangler CLI.

1. git clone https://github.com/n24q02m/wet-mcp && cd wet-mcp
2. wrangler login
3. Provision resources and apply the D1 schema:
   Terminal

   Copy

   wrangler d1 create wet-docs
   wrangler d1 execute wet-docs --file migrations/0001_init_wet.sql --remote
   wrangler d1 execute wet-docs --file migrations/0002_project_context.sql --remote
   wrangler d1 execute wet-docs --file migrations/0003_version_index_state.sql --remote
   wrangler vectorize create wet-docs-vectors --dimensions 768 --metric cosine
   wrangler kv namespace create wet-kv
   

   Paste the returned IDs into wrangler.jsonc.
4. Build the slim HTTP image from this checkout and push it directly to Cloudflare's managed registry (CF Containers cannot pull from external registries):
   Terminal

   Copy

   docker build --target http --build-arg SLIM=1 -t wet-mcp:beta .
   wrangler containers push wet-mcp:beta   # prints registry.cloudflare.com/<ACCOUNT_ID>/wet-mcp:beta
   

5. Set operator auth/storage and Browser Run secrets:
   Terminal

   Copy

   wrangler secret put CREDENTIAL_SECRET
   wrangler secret put MCP_JWT_SIGNING_SECRET
   wrangler secret put MCP_RELAY_PASSWORD
   wrangler secret put MCP_DCR_SERVER_SECRET
   wrangler secret put CF_BROWSER_RENDERING_TOKEN
   

6. wrangler deploy and complete setup in the browser relay form at your Worker domain.

Storage maps to Cloudflare via MCP_STORAGE_BACKEND=cf-kv (credentials/tokens, encrypted), DOCS_DB_BACKEND=cf-d1 (docs + BM25 full-text), and Vectorize (embeddings). The default headless renderer is BROWSER_BACKENDS=cf-browser-rendering. Local ONNX fallbacks are disabled in the slim image; configure search and cloud retrieval through each authenticated subject's relay record. Worker-wide search/model chains and provider keys are not forwarded to the container.

For a Cloudflare AI Gateway route, enter the following values in that subject's relay form (<CF_AIG_BASE> is the account/gateway base URL):

Store the matching OpenRouter/Cohere credentials and any keyed search-provider credentials (such as TAVILY_API_KEY) in the same subject record. All Wet synthesis and summaries use LLM_MODELS; there is no separate SUMMARY_MODELS field. This completion chain has no paid or alternate-model fallback. Cohere embedding/reranking and Browser Run may incur charges; obtain the required budget authorization before exercising them. Provision Vectorize and EMBEDDING_DIMS for a dimension supported by the selected embedding model. The earlier 768-dimension example is not a Cohere v4 compatibility guarantee.

Smithery

wet-mcp ships a smithery.yaml so it can be installed and run through Smithery. The manifest declares a stdio start command (uvx --python 3.13 wet-mcp) with an empty config schema -- no config is required to start, and providers and credentials are configured at runtime via the server's own config flow (see Configuration).

Trust Model

This plugin implements TC-Local (machine-bound, single trust principal). See mcp-core trust model for full classification.

License

Apache-2.0 -- See LICENSE.