skim

Token-efficient skim-then-expand I/O for agentic models (Claude Code / desktop Claude).

Independent community project — not affiliated with, endorsed by, or sponsored by Anthropic. No telemetry; runs entirely on your machine. See Trademarks, privacy & license.

Instead of reading a whole large file into context, the model calls skim_open(path) and gets a compact skeleton — structure, signatures, and (for logs/data) preserved critical values like ids, numbers, dates, and error codes — plus anchor ids. It reads the skeleton cheaply, then calls skim_expand(handle, anchors=[...]) to pull only the exact spans it needs, verbatim and lossless.

Compression is lazy, not lossy: nothing is paraphrased or destroyed, only deferred. Expansion is always one call away — which is what makes it safe for a closed model that can't ingest latent vectors.

What it does that other tools don't

The mainstream tools are lossy in context — repo-maps and --compress drop function bodies; summarizers and compression models paraphrase. The closest neighbors either cover code only, or show the model a transformed view and keep originals in a side cache. skim's line is stricter: whatever lands in context is verbatim source, and everything not shown is one anchored expand away — an agent can edit code from what it read without re-reading. That holds everywhere:

• Files — Python (ast) + ~17 languages (tree-sitter); bodies folded, one expand away.
• Whole repos — skim_repo builds a ranked, token-budgeted map; expand exact code from any file.
• Command output — skim_run compresses verbose test / build / log output, fully recoverable.
• Data & logs — a generic path with a retention guarantee (ids, numbers, error codes, negations are promoted into the skeleton, never silently dropped) and dedup of repeated blocks.

See DESIGN.md for the architecture and verified prior-art positioning.

Falsifiable, not claimed

A reduction percentage is marketing until you can check it. skim ships the checker:

uv run skim-verify path/to/your/gnarliest_file.py     # any file at all; exit code tells CI

Five invariants, verified on your files, locally: every non-blank line recoverable, expands byte-exact, anchors in-bounds, reconstruction equal to the decoded file, deterministic output. The test suite enforces the same contract with Hypothesis fuzzing on every path; a reproducible FAIL on any readable file is a bug — please report it. Summarize-first tools cannot ship this command, because for them the equivalent check fails by design.

The cost-vs-correctness question has its own open yardstick — eval/ACCURACY.md prices every eval question under a full read vs skim (including the rows where skim loses), and any other context tool can be scored under the same protocol.

Built to be audited

• Under 2,000 lines of stdlib Python for the whole package (the reading engine is ~1,200) — no ML models to download, no framework, no telemetry, dependency surface of one (mcp, plus optional tiktoken/tree-sitter extras). An afternoon's security review covers all of it: SECURITY.md is the threat model.
• A kill switch for every surface that isn't read-only: SKIM_RUN_DISABLED=1 removes the shell tool, SKIM_PATCH_DISABLED=1 removes the file editor; the readers keep working.
• Windows is a first-class platform, not a port: command output decoded as UTF-8 (no cp1252 mojibake), process trees actually killed on timeout (taskkill /F /T), CRLF preserved by skim_patch, and CI runs the full suite on windows-latest alongside Ubuntu.

Try the core (no install)

python demo.py path/to/file.py            # measured before/after token counts
python demo.py                            # demo on examples/sample.log (retention + dedup)

Results (hard data)

Every number below is generated live by uv run python benchmarks.py — full reproducible tables in BENCHMARKS.md. Measured on the running interpreter's standard library (real code); token counts via tiktoken cl100k_base (a proxy for Claude's tokenizer; ratios are tokenizer-robust).

Token savings on real code — 60 Python stdlib files: 387,911 → 122,660 tokens (68% fewer, 3.16×); per-file 1.5×–11×; 100% lossless, deterministic; ~linear runtime (~4 ms / 1k lines); pure CPU, no GPU, no network, no model calls.

Multi-language (tree-sitter, [lang] extra) — the same lossless engine on JS / TS / Go / Rust / Java / C / C++ / Ruby / PHP / C# and more. Example: a 481-line JavaScript module → 6.6× / 85% saved, lossless.

1:1 before/after on a real task (distribution-level) — "read the largest function in this module"; the model opens the file, reads the skeleton, expands exactly the one function it needs. Same answer, full skim cost (skeleton + expand) counted, across 24 large modules:

Per-task savings: median 51%, mean 51%, range 30–91% across 24 tasks — the honest distribution, not a cherry-picked best case. The win shrinks when you need most of a file.

Head-to-head — same 30 files (compare.py + real Repomix via npx):

skim is the only lossless option and uses 40% fewer tokens than Repomix --compress. The repo-map approach is ~3× smaller but discards bodies/docstrings/comments irreversibly. (Basemind is pure Rust and wasn't installed; its row reproduces the signatures-only mechanism — see COMPARISON.md.)

Correctness: 0 of 54,215 non-blank lines unrecoverable across 80 files; 202 tests / ~80% coverage with Hypothesis property fuzzing of both paths; an ~18,000-case adversarial campaign (every bug found is fixed and regression-locked). Reproduce: uv run python bench.py / pytest / benchmarks.py.

Run as an MCP server

The one-liner (installs from PyPI on first run, tree-sitter languages included):

claude mcp add skim -- uvx --from "skim-mcp[lang,tokens]" skim-mcp

Or from source:

git clone https://github.com/helloderekg/skim-mcp.git && cd skim-mcp
uv sync --extra lang                       # MCP SDK + multi-language (tree-sitter); drop --extra lang for Python-only

Register with Claude Code (use the repo's absolute path; forward slashes work on Windows):

claude mcp add skim -- uv run --directory /abs/path/to/skim-mcp skim-mcp

Or Claude Desktop — add to claude_desktop_config.json (%APPDATA%\Claude\ on Windows, ~/Library/Application Support/Claude/ on macOS), then restart:

{
  "mcpServers": {
    "skim": {
      "command": "uv",
      "args": ["run", "--directory", "/abs/path/to/skim-mcp", "skim-mcp"]
    }
  }
}

(If uv isn't found, use its full path from where uv / which uv.)

Tools

• skim_open(path, query="") → compact skeleton + anchor ids (read the skeleton; ids are in its expand("aN") markers). Pass query to also get matches (line + covering anchor) in the same call, no search round-trip.
• skim_expand(handle, anchors=[...]) → exact verbatim spans. Items are anchor ids ("a7") or literal line ranges ("L120-180") for when a grep already gave you line numbers.
• skim_search(handle, query) → which anchors/lines contain a string, without reading them.
• skim_run(command) → run a shell command, get a compact expandable view of its output (tests / builds / logs).
• skim_repo(path, query) → a lossless, ranked, token-budgeted map of a whole repo; expand exact code from any file. Ranked by query match when you pass one, else by import-graph centrality (PageRank over which files import which), so the load-bearing modules surface first.
• skim_patch(handle, anchor, new_text) → replace exactly one expanded span on disk, drift-safe: refused if the file changed since skim_open, LF/CRLF preserved, result re-verified from disk, fresh handle returned. Because expands are verbatim, an edit built from one applies safely — read 8% of the file, edit it anyway. (SKIM_PATCH_DISABLED=1 turns it off.)
• Spans are also MCP resources: @skim:skim://doc/<handle>/span/<anchor> pulls a span by reference.

What it looks like in practice