Donsetch README

The full upstream README, mirrored for reference. Install config, tool schemas, adoption signals and our own overview live on the Donsetch listing page.

DonSeTch — The web. On your terms. A pipe-smoking don against a red and black city skyline.

DonSeTch

The web tools your agent needs.
Search, fetch, crawl, and screenshot. One local Rust binary. No required API keys or accounts.

Latest release CI status Monthly npm downloads GitHub stars

Sponsor DonSeTch — see tiers and payment details

Install · Tools · Design · Proxies · Limits · Docs · Sponsor

Find a source, read the useful part, follow a link, check a claim. DonSeTch gives agents that web loop over MCP or from your terminal, with about 2,000 tokens of tool schemas.

Page reads launch no browser. Real client profiles, learned routes, and embedded V8 handle the fetch. Chromium is used separately for screenshots. Search needs no model download.

Start here

1. Install

Terminal
npm install -g donsetch
donsetch doctor

The installer downloads a SHA256-verified release binary. No Rust toolchain needed. Linux, macOS, and Windows are supported.

Homebrew, Pi, DeepSeek Harness, or build from source
InstallCommand
Homebrew · macOS/Linuxbrew tap dondai44423/donsetch && brew install donsetch
Pi native extensionpi install npm:donsetch
DeepSeek Harness plugindsh plugin --profile web add github:dondai44423/donsetch-dsh

Pi and dsh register native tools. For source builds, platform requirements, Docker, or installer troubleshooting, see the setup guide.

2. Connect your agent

Add this to your MCP client's server configuration, then restart the client:

config.json
{
  "mcpServers": {
    "donsetch": {
      "command": "donsetch",
      "args": ["mcp", "--supervised"]
    }
  }
}

Works with Claude Code, Cursor, OpenCode, and other MCP clients. Configuration formats vary; donsetch doctor --mcp prints registration examples using the binary's absolute path. --supervised restarts the stdio daemon after a crash.

Need an HTTP endpoint? Run donsetch mcp --http --port 8765 and connect to http://localhost:8765/mcp. HTTP setup and authentication →

3. Try it

server.ts
# Find sources.
donsetch search "rust async cancellation" --intent code

# Read the passages that answer.
donsetch fetch https://docs.python.org/3/library/asyncio-task.html \
  --focus "cancel a task"

# Map a documentation site.
donsetch crawl https://docs.python.org/3/ --mode map --topic asyncio

The CLI and MCP tools use the same engine. No account setup sits between install and first use.

Worth keeping in your toolkit? Star the repo. It helps other agent builders find it.

Four tools. One process.

Four noir illustrations: a retrieved document, a magnifying glass and globe, a connected street map, and a bellows camera.
ToolUse it forUseful controls
web_fetchPages, PDFs, structured site records, or up to 12 URLs at oncemust_contain, focus, toc, section, since_last
web_searchRanked sources and evidence snippets, keyless by defaultintent, lang, max_results, read
web_crawlA site model, a plan, or a bounded crawl with a digest and resumable jobmode, focus, max_pages, job, handle
web_screenshotA rendered PNG through an installed Chromium-family browserfull_page, wait_ms, deadline_ms

Your agent chooses what to read next. DonSeTch supplies the tools.

Read less. Keep the evidence.

Every view works over the extracted source, including PDFs and site adapters. You choose what comes back:

QuestionOptionResponse
“Does this page mention it?”must_containCounts, sections, and excerpts for up to eight phrases or regexes.
“Where is the answer?”focusRelevant passages, best first, with their section paths.
“What's in this document?”tocHeadings and section IDs, without the body.
“Read that part.”sectionOne section and its subsections.
“What changed?”since_lastChanges against your stored previous read, or unchanged.
“Keep going.”offsetContinue at the returned next_offset.
bash
donsetch fetch https://doc.rust-lang.org/std/vec/struct.Vec.html \
  --must-contain with_capacity --must-contain amortized

Amounts match by value: 15M can find EUR 15 000 000. An unread portion makes an absence check unverified; it does not become a false “not found.” Use --links when you need citation URLs.

Fetch views, search options, crawl budgets, and result formats →

Under the hood

Fetch flow: check cache, make a request with a coherent identity, classify the response, use a native interface or bounded V8 when needed, verify content, and remember successful routes. Mined requests replay over HTTP on later visits. Screenshots use a separate browser path.

Open the route map at full size →

The client stays consistent. TLS, HTTP/2, headers, hints, and cookies belong to one identity. A clearance is spent by the identity that earned it. Successful identities and routes are learned per site.

JavaScript runs when it has a job. A challenge can mint clearance in V8. An application shell can run its own program against a data-only DOM. Nothing is laid out or painted. When a successful content run reveals a reusable data request, the fetcher stores that request and replays it over HTTP on later encounters. A broken replay is discarded and re-learned.

Search ranks evidence. Sources are selected by query type, merged by measured index independence, and ranked with BM25+, RM3, and a compiled linear model for the query types where it won validation. No LLM, embeddings, or model download.

PDFs use text and pixels together. PDFium supplies glyphs and rendered pixels from the same document. DonSeTch uses both to recover reading order, tables, and forms. Scans and corrupt text regions can use optional OCR, with uncertainty marked.

Architecture, cache behavior, local state, and source map →

Measured, with the conditions attached

Recorded fetch run · 15 Nov 2026ContentMedian
Cold · 28 defended targets × 2 rounds · up to 3 attempts49/56 · curl 21/561.18 s
Learned state · same 28 targets × 3 rounds · no page cache · 3 s deadline69/840.98 s

These are home-IP measurements, not a promise for every site or network. The cold run paired curl in the same window. The learned run is a different protocol, not a controlled speed comparison. Neither run used the new program/replay transport.

Method, failures, and the separate live-web field test →

Proxies without the guesswork

You don't need one to start. A proxy is useful when an engine rate-limits your IP or a site treats your network differently.

bash
# Add one proxy. Use your own host and credentials.
donsetch proxy add 'socks5://user:pass@proxy.example:1080'

# Verify the tunnel, exit IP, and latency.
donsetch proxy check
donsetch proxy list

Restart the MCP server after changing the pool. The important part is what uses it:

ToolDefault with a configured poolChange it
SearchUses pool lanes for engine requestsAutomatic
CrawlStarts direct; later reads can spread across pool lanesdonsetch proxy crawl off disables pool use
FetchUses your own IPdonsetch proxy fetch on enables the pool

“Direct” here means outside the pool. A configured HTTPS_PROXY, HTTP_PROXY, or ALL_PROXY can still route that request. A failed configured proxy is reported; it does not silently fall back to your IP.

Crawl contacts the seed directly before pool lanes take over. To keep a site entirely out of scope, use a local deny rule.

HTTP/SOCKS5 formats, imports, credentials, environment proxies, and troubleshooting →

Optional keys

The default tools work without keys. Bring your own when you want an additional search provider or a paid fetch fallback.

bash
donsetch keys add tavily 'YOUR_KEY'
donsetch keys add exa 'YOUR_KEY'
donsetch keys default local           # Keyless first; paid search as fallback.

Native search providers: Tavily, Exa, Serper, SerpApi, SerpBase, Brave Search, TinyFish, Parallel, and Bright Data. Other providers can use the executable plugin interface.

Bright Data uses separate zones for SERP search and Web Unlocker:

bash
donsetch keys add bd 'YOUR_TOKEN::YOUR_SERP_ZONE'
donsetch keys add unlocker 'YOUR_TOKEN::YOUR_UNLOCKER_ZONE'

Unlocker runs after a local wall, only with a configured key. Attempts can cost money and are capped per day. Zone setup, costs, and troubleshooting →

Know the limits

SituationWhat happens
A page requires loginwall.auth. No sign-in, imported browser cookies, or stored login credentials.
A human challenge remainsA named wall. No local hCaptcha, reCAPTCHA, Turnstile, or press-and-hold solver. An optional Unlocker may help.
Content needs layout or canvasThe fetch stays a declared shell if the program cannot produce readable content. Use a screenshot for pixels.
A read hits its size or time limitPartial content or a named failure, with continuation information when available.
Search sources refuse the requestSource status and an empty or degraded answer. No invented results.
You repeatedly harvest a defended sitePacing helps; IP bans can still happen. This tool is for research, not bulk collection.

For signed-in browsing or long, human-paced sessions, use a real browser driver such as Bladebro.

Local processing still makes requests to the sites you ask for. Optional providers and proxies receive the traffic you route through them. Security policy →

Configuration and upkeep

bash
donsetch config show                  # Values and where they came from.
donsetch config show --markdown       # Every supported setting.
donsetch doctor --deep                # Live network and provider checks.
donsetch status                       # Version, keys, proxies, and cache.
donsetch update                       # Update from GitHub Releases.
donsetch rollback                     # Restore the previous version.

Settings live in donsetch.toml. Precedence: defaults → legacy environment → config file → DONSETCH_<SECTION>__<KEY>. Unknown keys and invalid values fail clearly.

Config paths, browser setup, Docker, HTTP transport, and troubleshooting →

Documentation

GuideWhat's inside
SetupInstall choices, platform matrix, MCP, HTTP, Docker, config, and repairs.
ToolsReading views, search, crawling, screenshots, BYOK plugins, and result contracts.
ProxiesPool routing, environment proxies, credential formats, and diagnostics.
ArchitectureFetch, search, extraction, PDF, state, and the code behind them.
MeasurementsBenchmark conditions and the live-web field report.
Local rulesRefuse a host before fetching and give the agent your instructions.
Bright DataOptional SERP and Web Unlocker setup.
ContributingDevelopment loop, tests, PRs, and releases.

Support the project

The DonSeTch don: top hat, pinstripe suit, pipe, and network smoke.
Good tools take work. Help keep this one independent.

If DonSeTch earns a place in your agent's toolkit, give it a star, share a useful example, or report a reproducible bug. Code and documentation contributions are welcome.

Support DonSeTch on Ko-fi

Sponsorship

DonSeTch is free and open source. Sponsorship pays for the time spent building it, maintaining the tools, and keeping releases coming.

TierMonthly pricePlacement in this READMEProduct integration
Silver$25/monthA text link to your site in the sponsorship section.—
Gold$79/monthA large banner linked to your site, with priority placement ahead of Silver links.A native DonSeTch integration may be considered when your product fits the tool. Scope and fit must be agreed before any integration is promised.

Gold integration depends on the product. A proxy service, search provider, or web-data product may be a good fit if it gives DonSeTch users something useful. Email with your product and proposed use; we'll discuss whether an integration belongs in DonSeTch and what it would involve. Gold includes the banner and link; an integration is conditional on that review.

How to sponsor

  1. Email bhandaribishesh879@gmail.com with your site or product URL, chosen tier, and how many months you'd like to sponsor. For Gold, include your banner if you have one and mention any integration idea.
  2. We'll confirm the placement, term, and any product-fit discussion.
  3. I'll share PayPal payment details and arrange the placement with you. GitHub Sponsors payouts aren't available in my region, so email is the reliable way to set this up.

Prepay to lock your rate

You can prepay any number of months at the current price. That rate stays fixed for the whole paid term, even if prices rise later.

ExampleTotalRate kept for
6 months of Silver$150All 6 months at $25/month
6 months of Gold$474All 6 months at $79/month

Later price changes apply to new sponsorships, not to an already prepaid term. One-time support is welcome at any amount through Ko-fi.

Using Bright Data? This partner link supports DonSeTch at no extra cost to you.


Built by Dondai and contributors.
AGPL-3.0-only · Changelog · Security · Issues