Ketch Cli

ketch

A stateless CLI for web search, code search, library docs, and scraping — one binary, no daemon, no API server to run.

Why ketch

Most research tooling for agents means wiring up several provider SDKs, each with its own auth and response shape. ketch collapses that into one binary with three research surfaces:

• ketch search — web search, no API key required (Brave, DuckDuckGo, SearXNG, Exa, Firecrawl, Keenable, Tavily, TinyFish, Parallel, SerpBase, Serply, or You.com)
• ketch code — grep real OSS source across public repos (Grep, Sourcegraph, or GitHub Code Search)
• ketch docs — curated, version-aware library documentation (Context7)

Plus ketch scrape and ketch crawl to turn HTML pages and text-based PDFs into clean markdown.

It's built for two audiences at once:

• Humans, who want a fast terminal tool for the same job curl | pandoc or a browser tab would otherwise do.
• AI agents, who want structured, predictable output (--json everywhere), documented exit codes for control flow, and a single ketch config call to discover what backends are active — no environment probing, no per-provider glue code.

An operator configures the backend once (ketch config set backend searxng); every agent invocation afterward just calls ketch search or ketch scrape without knowing or caring which provider is behind it.

Install

# macOS / Linux, any of x86_64 or arm64
curl -fsSL https://ketch.run/install | sh

# Homebrew
brew install ketch

# npm
npm install -g ketch-cli

# go install
go install github.com/1broseidon/ketch@latest

# Or download a prebuilt binary (linux/darwin/windows, amd64/arm64)
# from https://github.com/1broseidon/ketch/releases

The install script picks the right build for your OS and architecture, verifies it against the release's checksums.txt, and drops the binary in /usr/local/bin when that's writable, otherwise ~/.local/bin. Read it first if you'd rather not pipe to a shell: install.sh. Pin a version or change the target directory with sh -s -- --version v0.17.0 --bin-dir ~/bin.

Quickstart

$ ketch scrape https://go.dev/doc/effective_go
---
url: https://go.dev/doc/effective_go
title: Effective Go - The Go Programming Language
words: 16582
---
1. [Documentation](https://go.dev/doc/)
2. [Effective Go](https://go.dev/doc/effective_go)
...
## Introduction

Go is an open-source programming language that focuses on simplicity, reliability, and efficiency...

Search real OSS code with zero configuration:

$ ketch code "http.NewRequestWithContext" --lang go --limit 2
---
query: http.NewRequestWithContext
lang: go
backend: grepapp
result_count: 2
---
harness/harness  registry/app/remote/clients/registry/client.go  (line 207)
  req, err := http.NewRequestWithContext(ctx, http.MethodGet, buildPingURL(c.url), nil)
  https://github.com/harness/harness/blob/main/registry/app/remote/clients/registry/client.go
...

Web search works with zero configuration too — the default backend (auto) falls back through the keyless providers, so there is no key to set first:

ketch search "golang error handling"
ketch search "golang error handling" --scrape   # fetch + extract full content per result
ketch search "golang error handling" --multi    # federate across every usable backend, rank-fused
ketch search "golang error handling" --random  # pick one random backend, fallback to rest on failure

auto tries providers in a fixed order and returns the first that answers, reporting which one served in the backend: field. It prefers whatever you have actually configured — your own SearXNG or Degoog instance first, then any provider you have set a key for — and only then the keyless hosted providers. So setting a key is still how you get a specific provider and higher limits, and you do not have to also set backend for it to take effect:

ketch config set brave_api_key <key>   # auto now prefers Brave
ketch search "golang error handling" -b ddg   # or pick a provider explicitly

--multi queries several backends at once and fuses their rankings with Reciprocal Rank Fusion (a page several engines rank highly floats to the top), deduplicating by URL and tagging each result with the engines that returned it. --random shuffles the backend list, tries one, and falls back to the rest — ideal when you want one provider's results without wasting rate limits on all of them. Both support bare (all usable backends) or =brave,exa explicit lists, and both are mutually exclusive with --backend and each other. See the command reference.

Every command takes --json for structured output:

ketch scrape https://example.com --json
# {"url":"https://example.com","title":"Example Domain","markdown":"..."}

Pipe any HTML through ketch's extraction pipeline without a fetch:

curl -L https://chain.sh/ketch | ketch extract
cat page.html | ketch extract --select article --max-chars 4000

Extraction

Extraction reads the page's own structure: the content landmark it declares (main, article), a document assembled from uniform sections, or the smallest element holding its prose. Site furniture is removed by what it is — navigation, hidden and collapsed controls, link rails, tables of contents — and readability is the fallback for a page that declares no structure. The extract_mode config key sets what pruning may drop: clean (the default) also drops blocks by name and phrase — related-post rails, comment threads, share bars, "was this helpful?" boxes — for the leanest markdown, and complete keeps everything the structure does not condemn, trading a little precision for the last fraction of recall. Pages served in a legacy encoding are decoded before extraction.

PDF extraction

ketch scrape detects PDFs from the response MIME type or %PDF- signature and extracts their text with a built-in pure-Go parser:

ketch scrape https://example.com/report.pdf

Scanned/image-only PDFs need OCR and return a precondition error with an OCR-converter hint in the built-in path. Operators can configure an external PDF-to-Markdown converter that writes Markdown to stdout (capped at 10 MiB); its shlex-parsed command must contain exactly one {input} placeholder:

ketch config set external_pdf_to_md_converter_command 'pdftotext "{input}" -'
ketch config set external_pdf_to_md_converter_timeout_sec 300

When configured, the external converter is authoritative: failures are returned rather than silently falling back to the built-in parser. PDF binary output is never emitted: --raw and --select reject PDFs as validation errors. With --force-browser, PDF markdown still uses text extraction and never opens Chromium's PDF viewer.

Tags — bookmarks for agent workflows

Save useful sources while you research a project, then find them again in a later session. Tags group documentation, code references, search results and write-ups under the same project or topic. One source can belong to several tags.

ketch search "guacamole ldap authentication" --scrape --tag remote-access
ketch code "guacamole ldap" --tag remote-access
ketch docs "apache guacamole" --tag remote-access
ketch tag add remote-access https://example.com/read-this-later  # no network

ketch tag show remote-access
ketch tag show remote-access --limit 10 --json
ketch tag show remote-access --limit 0       # all entries
ketch tag remove remote-access https://example.com/read-this-later

--tag works on search, code, docs, scrape and crawl, including --no-cache. A bookmark keeps the source URL, a bounded title and description, and the time it was tagged. Descriptions come from search results, snippets or fetched content; full bodies belong to the separate page cache.

tag show returns the newest 50 entries by default, ordered by tagging time and then URL for ties. Use --limit N to change that, or --limit 0 for all. Text output says showing N of M when limited; --minimal keeps its TSV output and puts that notice on stderr. JSON and MCP return entries and cached totals for the whole tag, shown for the returned pages, and cache_status. The MCP tag tool accepts the same limit on operation: show.

Read the index, choose a source, and scrape its URL. cached: false usually means there is no fresh local body, not that the link is broken. If the page cache is locked or unreadable, bookmarks still work: cache_status: unavailable means the cached: false flags are unverified. Fetches can proceed without the page cache. A bookmark does not guarantee an upstream page still exists.

Bookmarks survive cache expiry and cache clear. Clear removes page bodies and deletes the cache file, returning the space; expired pages are also swept automatically, so the file stays near the size of what is fresh. tag add accepts absolute http(s) URLs without fetching them (anything else is bad input, exit 2), preserves existing metadata on a cold re-add, and fills missing titles and descriptions on a later fetch. Removing a bookmark uses the displayed URL, regardless of cookies, User-Agent or URL rewrite settings. Nothing expires bookmarks; use tag remove to tidy up — with URLs, it reports the ones that were not under the tag as missing.