vs-token-safer

English · 한국어

A token-saving code layer for Claude Code, for any codebase. TypeScript, JavaScript, and Python work with zero setup; C# and C++ add a language server; and 30+ more languages come in on the built-in tree-sitter tier. Proven up to a 26k-translation-unit Unreal Engine monorepo — so your web, backend, or data repo is the easy case.

Your coding agent has a small context window. Your repo is large. vs-token-safer sits in between.

Ask where something is, what calls it, or even "how does the auth flow work?" when the name escapes you — and instead of pasting a wall of source into the chat, it replies with a short file:line list.

• Builds normally? The answers are exact — it reads the same code index your editor relies on.
• No toolchain set up? It still locates your functions and classes.
• Forgot the name, only remember what it does? It finds it from the vocabulary your own code already uses — no AI model, nothing uploaded.
• Markdown or a config file? Jump straight to one section by its heading instead of opening the whole file.

A companion plugin does the same for giant editor and build logs. None of it leaves your machine.

# Claude tries to grep code → the hook REWRITES it to the indexed query, in place:
$ grep -rn "createSession" src/
↻ [vs-token-safer] Rerouted → search_symbol "createSession"   # semantic, not a text match
  func createSession (in AuthService)  @ src/auth/session.ts:142   (+2 more)
  → ~120 tokens   (grep would have dumped thousands of lines)

# Editing that symbol? Name it — no Read-the-whole-file, no line counting:
$ replace_symbol_body symbol="createSession" body="…"        # preview; apply=true writes
  replace_symbol_body "createSession" — PREVIEW at src/auth/session.ts:142-160

_{Same flow on TypeScript, Python, C#, C++, Go and more (clangd · Roslyn · tsserver · pyright · tree-sitter). VTS_REWRITE=0 blocks instead of rewriting.}

_{The built-in dashboard (vts serve) — your indexed repo as a live 3D graph, all on 127.0.0.1. Three modes: include graph (clangd), call graph (live LSP call hierarchy by symbol), and symbol graph — a tree-sitter map of files + import edges that needs no language server, so it renders on any of the 17 tree-sitter languages with zero setup.}

Why

• Keep the context lean. grep on a large repo — a TypeScript/Python monorepo, a C#/.NET solution, even a 26k-TU Unreal C++ tree — floods the context. The language-server index stays token-capped — ~97–99% smaller (benchmarks).
• Claude keeps reaching for grep. The hook doesn't just block it — it rewrites the command to the indexed query in place, so the search still runs and the flow never breaks.
• Edit by symbol, not by line. Replace/insert-around/delete a declaration by naming it — the index supplies the span, so you skip reading the whole file into context.
• Review a diff by its blast radius, not by eye. detect_changes maps a git diff to the symbols it changed, walks each one's callers, and scores the risk LOW/MED/HIGH — from the blast radius, the caller-cascade depth, and which historically co-changed files this diff left out. A review starts from "what does this actually touch," capped to file:line, nothing uploaded. It's the code-review-context idea done the vs-token-safer way: the official language server for the blast radius and your own git history for the coupling — no persistent graph database, no embeddings.
• You can't tell how much grep still slips through. vts discover reads your recent sessions and reports exactly which searches bypassed the index and what they cost.
• The language server runs headlessly — no editor open, unlike an IDE-proxy approach.

Quickstart

# 1) Install (also auto-installs the gamedev-log-analyzer sibling)
/plugin marketplace add JSungMin/vs-token-safer
/plugin install vs-token-safer@vs-token-safer
/reload-plugins        # first run auto-installs the server deps (no manual npm)

# 2) Configure — detects the backend, asks for the project path, writes the config
/vs-token-safer:setup

Then restart the Claude Code session (the vs-search MCP server only starts on a fresh session). Verify the tools appear and that grep src/**/*.cpp is rerouted to the index. Prerequisites: Node ≥ 18 and a language server — clangd (C/C++) / Roslyn (C#) you install; JS/TS + Python auto-install. Details in Prerequisites below.

Want only the log analyzer? /plugin install gamedev-log-analyzer@vs-token-safer.

How it works

vs-token-safer isn't a search box — it's a precision ladder. You ask where something is, what calls it, or "how does the auth flow work?" when the name escapes you, and it answers at the highest precision it can reach, then tells you which rung the answer came from:

• EXACT — you know the name and the project builds → the official language server (clangd / Roslyn / tsserver / pyright), the semantic ground truth.
• SYNTACTIC — no toolchain set up → a tree-sitter parse (36 grammars bundled, no native build; 19 languages with tuned declaration extraction, the rest via a generic parse) still returns real declarations, not a grep.
• FUZZY — you only remember what the code does → a concept dictionary mined from the repo's own identifiers + comments (no AI model, nothing uploaded).
• SECTION — it's a doc or config, not code → Markdown / TOML / YAML / CSS / HTML addressed by heading.

The rungs aren't a one-time pick — they connect, and vts switches between them as it learns more. Start on FUZZY when you only know the intent; the moment concept_search surfaces a real name, vts climbs to EXACT to confirm it against the semantic index (and an exact search that misses drops back down to FUZZY). The hooks steer that hand-off in both directions, so "I don't know the name" turns into a precise, semantically-verified file:line instead of a dead end.

Graceful degradation — a locate never hard-errors on a cold or missing toolchain. When clangd is missing, cold, or still indexing, a locate degrades to the SYNTACTIC (tree-sitter) tier with a one-line advisory instead of erroring or blocking, then climbs to EXACT once clangd warms. If a committed .vts-index has drifted from the source, the answer is still served but labeled SYNTACTIC · STALE (with a climb: vts index hint). And on a cold locate over a large tree, vts auto-builds the .vts-index/ in the background so the next query (or the same session, once the build finishes) is instant — you never wait on it. So vts index below is best thought of as auto-built in the background; commit it to share, not a manual step you have to remember.

Every answer comes back capped to file:line (never source bodies) and carries a one-line completeness certificate naming the rung — so the model always knows whether it got the semantic truth or a fallback. On a 3-language, 150-file benchmark that's 87% fewer tokens than grep (~138× on a real Unreal Engine 5 tree). Underneath, four mechanisms make Claude actually use the ladder instead of reaching for grep:

Engine = official, glue = ours. clangd (LLVM) and Roslyn (Microsoft) do the analysis; this repo only writes the LSP↔MCP glue. No third-party MCP server runs over your source. Local-only, nothing uploaded.

Tools

All search/edit goes through an official language-server index — clangd (C/C++), Roslyn (C#/.NET), tsserver (JS/TS), pyright (Python) — and comes back as a compact, capped file:line list (never source bodies). MCP server vs-search; same tools as the vts CLI.

Search / navigate

Edit (symbol-level — name it, don't line-count) — preview by default, apply=true writes.

Docs & config too (structure tier). Point any of document_symbols / read_symbol / replace_symbol_body / insert_symbol / safe_delete at a Markdown / AsciiDoc / reST / TOML / INI / YAML / JSON / text file and the "symbol" is a section (heading, [section], or key): outline a 2000-line CLAUDE.md in ~30 lines, read or replace one ## Section by name without Reading the whole file. No language server, no new tools — same token-safer move, for documents.

Admin / meta — one MCP tool vts_admin {op, params} (folded to keep the per-session tool-definition cost small; the CLI keeps the bare subcommands):

e.g. vts_admin {op:"git", params:{argv:["status","-s"]}}. Or hand a whole "where is X / what calls Y / find file W" lookup to the code-locator subagent — it searches in its own context and returns only the file:line table.

Dashboard — vts serve. A local, interactive view of what vts knows + how much it saved: the savings trend, language mix, per-tool savings, and an interactive 3D graph (WebGL / Three.js) with two modes — the include graph (files sized by include fan-in) and an on-demand call graph (type a symbol → its transitive callers/callees, traced live through LSP callHierarchy — no persistent index; shows call counts per node/edge). Nodes are laid out on a spherical shell (so they spread out, not clump); drag/WASD to orbit, wheel/+-- to zoom, R to fit, hover for file:line. Symbol search has live autocomplete (/symbols); color by connected-component groups · repo (which repository each node is from, with a legend) · heat; click a node to drill into its group (Esc/Backspace to pop out); a focus/maximize toggle, a highlight filter, and a node/edge metrics overlay.

Easiest via the slash commands: /vs-token-safer:viz (open) and /vs-token-safer:viz-stop (close). Or the CLI:

vts serve --open     # → http://127.0.0.1:8731/  (launches the browser; --port N to change)
vts serve --stop     # stop it (or Ctrl-C the process)

It's 127.0.0.1-only and serves a fully self-contained page — CSS/JS inlined and Three.js vendored locally (server/vendor/, served same-origin, never a CDN), so nothing leaves the machine; it renders with the network unplugged. Same trust model as the rest of vts. Built on Node's stdlib http (no web-framework dependency), and it runs only when you invoke it — the MCP server never starts it, so the steady-state package stays a thin stdio client. The 3D graph caps at VTS_VIZ_MAX_NODES (200) for smoothness.

$ vts symbol --q createSession --projectPath ./app
3 symbol(s) matching "createSession" (backend: typescript, root: ./app):
func createSession (in AuthService)  @ app/src/auth/session.ts:142
method createSessionToken (in TokenStore)  @ app/src/auth/token.ts:88
func createSessionCookie  @ app/src/http/cookies.ts:31

✓ Saved ~4,200 tokens here (96.8% / 31× smaller than the raw index response).

The two plugins

vs-token-safer declares gamedev-log-analyzer as a dependency, so one install pulls in both. Used together: the log analyzer emits file:line per entry → hand it to goto_definition/find_references to open the code, without grepping or dumping the raw log. The handoff runs in reverse too — a code search aimed at a log (Logs/, .log/.jsonl) points you back at gamedev-log instead of an empty result.

Companion: drive vs-search with a local model

vts-local-orchestrator — a separate, optional companion that lets any local LLM (Ollama, full-GPU — model-agnostic, default gemma4:e4b, chosen by benchmark) drive these same vs-search tools. Claude delegates cheap, high-volume code-location to the free local model and receives only the compact file:line answer, so the raw search output never enters Claude's context.

• CLI (qvts) + a live web dashboard with a 3-way token-savings panel (this method vs CC-using-vs-search vs CC-using-grep), plus a delegation-routing skill for Claude Code.
• Token-savers: a persistent savings ledger (qvts --savings), a locate cache (zero-cost repeats), batch delegation (--batch), and terse repo-relative file:line output.
• Fully local — nothing transmitted off-machine, same as this plugin's charter.

Install: clone it next to this repo, npm install, then bash setup-macos.sh (Windows: setup.ps1). See that repo's README for the pipeline, the model benchmark, and the savings model.

Performance

The token win scales with repo size and is language-agnostic — a TypeScript service or a Python codebase sees the same shape (the deterministic 3-language benchmark cuts ~87%; details in BENCHMARK.md). The most extreme case we've measured is a large Unreal Engine 5 project: finding one public engine symbol (FGameplayTag) via Bash grep-and-paste vs this plugin. No project source is reproduced, only aggregate counts.

~99.3% fewer (~138×). grep returns the full text of every matching line and matches by text (comments, strings, unrelated identifiers); the plugin returns one file:line per semantic hit, capped. The mock-LSP eval (node eval/run.mjs, no toolchain) gates this on every commit: ~57,308 → ~1,515 tok = 97.4% (92 checks).

Syntactic tier vs clangd — same location, no cold wait. On a second real UE5 game module (3,143 files, 133,890 symbols; aggregate counts only), the tree-sitter tier returned the same file:line as clangd's EXACT answer on every sampled symbol (line delta 0), in ~240 ms vs clangd's ~127 s cold-to-first-answer — a ~530× gap the semantic tier doesn't close for a locate. That's why the ladder answers tree-sitter-first while clangd warms, then climbs to EXACT. One-time index build ~62 s; details in BENCHMARK.md.

vts scope --projectPath /path/to/UE        # shows current scope, kept/total TUs, and top-level dirs to pick
vts setup --scope "MyGame,Plugins"         # persist it (or set VTS_SCOPE="MyGame,Plugins"); then reload/restart

vts preindex --projectPath /path/to/UE     # honors the scope above

vts index --projectPath /path/to/repo      # writes .vts-index/symbols.jsonl (commit it!)
vts index --status                          # show the current committed index

vs-token-safer savings (local, 1 search(es))
  total saved: ~4,200 tokens vs forwarding raw index responses
  raw → output: 4,340 → 140 tok (~31× smaller)
  est. value: ~$0.01 (@ $3/Mtok — set VTS_USD_PER_MTOK)

$ vts discover --since 1
  86 code search(es) bypassed vts (Grep×48, Glob×18, grep×12, find×8)
  catch-rate: ~770,333 tok caught (via vts) vs ~28,692 still bypassing → 96.4% routed through vts

Prerequisites (details)

git clone https://github.com/JSungMin/vs-token-safer
cd vs-token-safer/server && npm install && npm link   # provides `vts`
# or run directly: node /path/to/vs-token-safer/server/cli.js symbol --q SpawnActor --projectPath /path/to/proj

Configuration

/plugin marketplace update vs-token-safer
/plugin update vs-token-safer
/reload-plugins
# then RESTART the session — REQUIRED.

Status & safety

• clangd & Roslyn live-verified — search_symbol/find_references/goto_definition confirmed against real clangd (incl. a real Unreal 5.x game project end-to-end) and Microsoft.CodeAnalysis.LanguageServer. Needs clangd ≥ 22 and a correct compile DB.
• Local-only, nothing uploaded. The hook only inspects the command string (honors VTS_ENFORCE=0); the language server runs over stdio; the only outbound call is the first-run npm install of the MCP SDK. It writes only its config + a local savings ledger under ~/.vs-token-safer/. See SECURITY.md and PRIVACY.md.
• Savings/benchmark numbers are response-shaping (raw index → capped); savings vs grep are larger (BENCHMARK.md).

Contributing

Issues and PRs welcome — bug reports, new backends/engines, language mappings, docs. Keep PRs small, evidence-backed, and free of proprietary data (real paths/symbols/project IDs); add an eval/run.mjs guard for any new code path. See CONTRIBUTING.md. If this saved you tokens, a star helps others find it. ⭐

Acknowledgments

vs-token-safer stands on ideas from the open-source code-intelligence community. With gratitude to:

• codebase-memory-mcp (DeusData) — the tree-sitter tags.scm call-site approach, the multi-hop call-hierarchy (trace_path) shape, and content-hash-keyed caching. Our difference: we keep the official language server as the semantic source of truth and use tree-sitter only as a zero-setup syntactic tier below it — no reimplemented type-resolution layer, no persistent semantic DB; everything stays local and nothing is transmitted.
• Codeix (montanetech) — the idea of a plain, git-committable JSONL symbol index. Our difference: ours is a cold-start accelerator that a language server automatically supersedes once it has indexed.
• Code Context Engine (elara-labs) — the token-savings framing for AI code search. Our difference: no embeddings/vectors (so no nearest-but-wrong retrieval) — exact file:line, token-capped.
• Serena — symbol-level editing (replace_symbol_body / insert_symbol / safe_delete), here layered on the LSP with preview-by-default.
• The tree-sitter project and tree-sitter-wasms for the prebuilt grammars that power the syntactic tier.

Each of these made vs-token-safer better. Thank you. (Reuse here always keeps our charter: official engines do the analysis, output is token-capped file:line, and nothing leaves your machine.)

References & related work

The design choices here have academic backing, and the field is moving fast. The companion papers carry the full treatment — paper/vs-token-safer.tex and the fuzzy-retrieval follow-up paper/fuzzy-concept-dictionary.tex. Key references:

Foundations we build on

• HyperAgent — Phan et al., arXiv:2409.16299. Generalist software-engineering agents; motivates giving the model structured tools instead of raw grep.
• Codebase-Memory: Tree-Sitter Knowledge Graphs for LLM Code Exploration via MCP — arXiv:2603.27277. The closest sibling. Our difference: official-LSP ground truth + capped file:line, no persistent semantic DB, nothing transmitted.
• Dead-code lineage — Rapid Type Analysis (Bacon & Sweeney, OOPSLA'96) and demand-driven reachability (the Go deadcode model) underpin vts dce's preview-only call-graph reachability.

Related & subsequent work (2024–2026) — two of these are now migrated into vs-token-safer (marked Migrated).

• LARGER: Lexically Anchored Repository Graph Exploration and Retrieval — Hu et al., arXiv:2605.16352. Formalizes "lexical anchor → structural expansion" — the academic shape of our fuzzy→climb-to-exact ladder, also without embeddings. Migrated → concept_search now expands the import-graph neighbourhood only from high-confidence lexical anchors (a neighbour lifts a symbol only if its own match clears a fraction of the strongest one; VTS_CONCEPT_ANCHOR_MIN), so a weak/cross-cutting neighbour can't drag its imports up the ranking.
• Pseudo-relevance feedback (RM3) — Lavrenko & Croft, SIGIR 2001 (the classical embedding-free retrieval-feedback technique). Migrated → concept_search runs a second pass that mines expansion terms from the top results' own identifiers + comments and re-scores — bridging a synonym the query couldn't reach lexically ("warm" → the warmset's warming/dominant; "reachable" → the dead-code module's fixpoint/cascades). Drift-guarded (terms need a ≥2-result consensus, idf-ranked, capped) and the climb seed stays the pre-feedback exact match, so it widens recall without chasing the feedback. VTS_CONCEPT_PRF=0 reverts to single-shot.
• Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient? — Hsu, Yang & Lin, arXiv:2605.10848. Empirical case that lexical retrieval suffices inside the agent loop — backing for our no-embeddings charter.
• One Tool Is Enough: RL for Repository-Level LLM Agents (RepoNavigator) — Zhang et al., arXiv:2512.20957. A single jump-to-definition tool, RL-trained, beats large multi-tool agents — a strong recent endorsement of LSP navigation over grep.
• DCE-LLM: Dead Code Elimination with Large Language Models — arXiv:2506.11076. LLM-judged dead code. Our difference: vts dce stays preview-only static reachability — deterministic, no model judgment, with safe_delete as the disposal backstop.
• cAST: Structural Chunking via Abstract Syntax Tree — Zhang et al., arXiv:2506.15655. Tree-sitter chunking for code RAG — the neighbor of our SYNTACTIC tier. Migrated → read_symbol now cuts an over-budget body at a whole-child AST boundary (the end of a complete member/statement), never mid-statement, so the returned source stays syntactically whole (falls back to the plain line cap when tree-sitter is absent).
• RepoGraph (arXiv:2410.14684) and LocAgent (arXiv:2503.09089) — repository code-graph localization; we reach the same call/dependency structure via live LSP call-hierarchy, with no prebuilt graph and no embeddings.

License

MIT © 2026 JSungMin