Vex README

Vex

Fast hybrid structural + semantic code search. Vector + index.

Why Vex? · How It Compares · Installation · Quick Start · Commands · Configuration · How Search Works · Benchmarks · Supported Languages · Integration · Testing · Architecture

$ vex check "TelemetryProcessor"           # 4ms — does it exist? where? (exact name)
$ vex show "TelemetryProcessor"            # extract the class body (not the whole file)
$ vex usages "Config" --strict             # who references this symbol? (binder-resolved, no noise)
$ vex callers "process_event"              # who calls this function? (~4ms; covers module-scope + Python/Java decorators)
$ vex implementations "BaseService"        # who extends/implements this?
$ vex search "timeout retry"               # fuzzy / multi-word — BM25 finds rare body terms
$ vex search "handle alert" --semantic     # find by meaning, not just name
$ vex pattern 'fn $NAME($$$) -> Result' --lang rust    # AST pattern matching (like ast-grep)
$ vex similar "PaymentService"             # semantically close symbols
$ vex duplicates --threshold 0.95          # near-duplicate pairs
$ vex bundle --mode symbol --symbol Foo    # body + callers + callees + similar in 1 call

Pick the right tool: vex check for "does Foo exist?", vex search for "find me something about retries". search is a ranked blend — it surfaces neighbors (callers / imports) when no symbol literally matches, which is great for exploration and wrong for exact-name lookup. v1.15.0 prints a stderr hint when an identifier-shaped search returns 0 FST hits.

Why Vex?

• ~4-5ms search after indexing — FST-based O(query_len) lookup, not O(symbols); constant regardless of project size. Requires a pre-built index. Indexing is a one-time cost (hundreds of ms on typical projects) and builds more than a plain text index — FST + BM25 + call graph + type-hierarchy + trigram skip-index — so it trades a slower build for far cheaper, richer queries (see Benchmarks)
• 3-channel hybrid search — structural FST (names) + BM25 (rare body terms) + semantic HNSW (meaning), fused via Reciprocal Rank Fusion. Find symbols when you don't know the exact name AND when generic semantic-only search would be too noisy
• Persistent call graph — vex callers/vex callees read from a persistent index built at index time (~4ms), not a live tree-sitter scan (seconds): callers is a name-keyed FST, callees is a dense CSR index (v9+). Module-scope expressions are reported via synthetic <module:path> callers (Phase 14.1); Python + Java function/method decorators (Phase 14.2), Kotlin annotations + C# method/constructor attributes (Phase 14.2.2), and TypeScript method decorators + Rust outer attributes (Phase 14.2.1) emit forward edges to their targets. Class-level decorators remain invisible — see docs/LIMITATIONS.md
• Pluggable embedder — Embedder trait + registry; swap MiniLM-L6-v2 for future code-specific models (BGE, CodeBERT) without touching call sites
• Token-efficient — compact output saves typically 6x fewer tokens than grep on average lookups (up to 217x on minified JS/CSS); vex show extracts just the symbol body instead of the whole file
• 19 languages indexed via tree-sitter, with three coverage tiers: type-aware --strict usages on 8 binder languages (Rust / TypeScript / Python / C# / C++ / Go / Java / Kotlin); indexed pattern prefilter on 15 T1+T2a languages; baseline structural + semantic search on all 19 (see Supported Languages for the matrix)
• Single binary, zero config — no LSP servers, no databases, no Docker. Just vex index && vex check Foo

What Vex isn't

vex is a static-analysis indexing tool, not a language server. Set expectations honestly:

• Not an LSP replacement. No go-to-definition into third-party packages, no rename refactoring, no type-checking, no hover docs. For those, keep your LSP.
• vex search is a ranked blend, not an exact-name lookup. Structural FST + BM25 + semantic fused via RRF return relevance-ordered results — when no symbol literally named Foo lives in the index (imported from a dependency, deleted, typo), BM25 may surface callers / imports as if they were the definition. For exact-symbol questions ("does it exist?", "show me the body", "who calls it?") use vex check Foo / vex show Foo / vex usages Foo --strict — they bypass the ranker. v1.15.0 prints a one-line stderr hint when an identifier-shaped query gets zero FST hits.
• No dynamic-dispatch visibility. Decorator routing (@router.get("/path")), string-resolved factories (uvicorn.run("main:app")), reflection (getattr(obj, name)()), and macro-expanded references are all invisible to every vex command. vex grep '\bname\b' is the textual escape hatch.
• vex callers has uneven coverage outside function scope. Module-level expressions like app = create_app() are reported via synthetic <module:path> callers (Phase 14.1). Python + Java function/method decorators (Phase 14.2), Kotlin annotations + C# method/constructor attributes (Phase 14.2.2), and TypeScript method decorators + Rust outer attributes on fns/methods (Phase 14.2.1) emit forward edges — vex callers GetMapping lists every Spring handler, vex callers HttpGet every ASP.NET action, vex callers test every #[tokio::test]. Class-level decorators (14.6) remain on the roadmap.
• vex usages quality varies by language. 8 binder-supported languages get refactor-grade --strict refs; the other 11 use an identifier scanner with a higher false-positive rate.

See docs/LIMITATIONS.md for the full coverage matrix, concrete repros, and recommended workarounds per query type. Read it before evaluating vex on a Python/FastAPI/Django codebase — the framework patterns are the most-flagged gaps.

How It Compares

Note: vex search speed assumes a pre-built index. Ripgrep and ast-grep require no upfront indexing and work immediately on any directory. The tradeoff is amortized: if you search the same codebase many times (typical in agent workflows), the one-time indexing cost pays for itself.

Best for: fast symbol search in AI agent workflows where token efficiency matters. Not a replacement for LSP-based tools (no refactoring, no go-to-definition in dependencies).

Installation

# Install script (macOS arm64, Linux x86_64): vex + vex-mcp into ~/.local/bin
curl -fsSL https://raw.githubusercontent.com/tenatarika/vex/main/install.sh | sh

# Windows (PowerShell): vex + vex-mcp into %LOCALAPPDATA%\Programs\vex, added to PATH
irm https://raw.githubusercontent.com/tenatarika/vex/main/install.ps1 | iex

# Homebrew (macOS/Linux)
brew tap tenatarika/tap
brew install vex

# cargo-binstall: prebuilt release binaries, no compiler needed
cargo binstall vex-search                # → ~/.cargo/bin/vex
cargo binstall vex-search-mcp            # → ~/.cargo/bin/vex-mcp (MCP server)

# crates.io (compiles from source; the crate is `vex-search`, the binary is `vex`)
cargo install vex-search --locked        # → ~/.cargo/bin/vex
cargo install vex-search-mcp --locked    # → ~/.cargo/bin/vex-mcp (MCP server)

# From source (any platform with a Rust toolchain)
git clone https://github.com/tenatarika/vex.git
cd vex
cargo build --release
cp target/release/vex ~/.local/bin/

The install script takes VEX_VERSION=1.27.4 to pin a release, VEX_INSTALL_DIR to install elsewhere and VEX_NO_MCP=1 to skip vex-mcp. On other platforms, musl or glibc older than 2.34 it stops and points to a source build. cargo binstall uses the same release archives where they exist and builds from source elsewhere; on Windows it installs vex.exe without the archive's DirectML.dll, so GPU indexing falls back to CPU.

What cargo install vex-search (and any source build) needs:

• Network at build time: the build downloads a prebuilt ONNX Runtime. Prebuilts exist only for aarch64-apple-darwin, x86_64/aarch64-unknown-linux-gnu and x86_64/aarch64-pc-windows-msvc; on any other target (Intel macOS, musl, …) point ORT_LIB_LOCATION at a local ONNX Runtime build.
• A C/C++ toolchain (Xcode Command Line Tools, build-essential, or MSVC Build Tools) for the tree-sitter grammars. GCC must be 12 or newer: GCC 11 (the default on Ubuntu 22.04) can't compile the bundled numkong vector kernels. Install gcc-12 g++-12 and build with CC=gcc-12 CXX=g++-12 cargo install vex-search --locked.
• Linux: libssl-dev and pkg-config (Fedora: openssl-devel). The HTTP stack links OpenSSL through native-tls.
• The first vex index --semantic downloads the ~86 MB embedding model; structural search needs no download.
• vex-search-mcp only installs the MCP server. It runs the vex CLI, so install vex-search too and keep vex on PATH, or set VEX_BIN to its full path.

Linux

Pre-built vex ships in every GitHub Release for x86_64-unknown-linux-gnu. It needs glibc 2.34 or newer (Ubuntu 22.04+, Debian 12+, RHEL / Rocky / AlmaLinux 9+); on older systems build from source:

curl -fsSL https://raw.githubusercontent.com/tenatarika/vex/main/install.sh | sh
# or by hand:
curl -L https://github.com/tenatarika/vex/releases/latest/download/vex-x86_64-unknown-linux-gnu.tar.gz | tar -xz
mv vex ~/.local/bin/      # or: sudo mv vex /usr/local/bin/
vex --version

Built on the ubuntu-22.04 GitHub runner (glibc-linked, libstdc++ linked in). For older glibc distros, musl-based distros (Alpine, NixOS without nix-ld), or aarch64 Linux (Graviton, Pi 5, Ampere) — build from source via cargo build --release.

Windows

Pre-built vex.exe ships in every GitHub Release. The quickest way to install it, from PowerShell:

irm https://raw.githubusercontent.com/tenatarika/vex/main/install.ps1 | iex

It puts vex.exe, vex-mcp.exe and DirectML.dll in %LOCALAPPDATA%\Programs\vex (a per-user folder) and adds it to your user PATH. VEX_VERSION, VEX_INSTALL_DIR and VEX_NO_MCP=1 work as in install.sh. To install by hand instead:

1. Download vex-x86_64-pc-windows-msvc.tar.gz from the latest release
2. Extract vex.exe somewhere stable (e.g. C:\Users\<you>\bin\) — tar -xzf vex-x86_64-pc-windows-msvc.tar.gz from a recent PowerShell, or 7-Zip / WinRAR via right-click. Security note: vex.exe loads the bundled DirectML.dll from its own folder, so on a multi-user machine or shared drive prefer a directory other users can't write to (e.g. C:\Program Files\vex\ — with the trade-off that vex self-update then needs an elevated shell). See GPU_SUPPORT.md §6.
3. Add that folder to PATH (System Properties → Environment Variables → edit Path → add the folder)
4. Open a fresh terminal and run vex --version

To update, run vex self-update — it fetches the latest release, picks the right archive for your platform, verifies its signature, and replaces the binary in-place. On Windows it also installs/refreshes the bundled DirectML.dll sidecar (skipped when byte-identical; re-installed if an older self-update dropped it — updaters up to v1.16.0 extracted only the binary). Same command works on macOS and Linux too.

GPU acceleration is built into the prebuilt binaries — Windows ships with DirectML (any DX12 GPU, driver-only; the redist DirectML.dll is bundled in the archive) and macOS arm64 with CoreML. NVIDIA CUDA is a source-build opt-in. Run vex gpu to check, and see GPU Acceleration.

Quick Start

# Index a project (structural only — fast)
vex index --path /path/to/project

# Index with semantic embeddings (slower first time, downloads 86 MB model)
vex index --path /path/to/project --semantic

# Exact-name lookup (does this symbol exist?)
vex check "PaymentService"

# Extract a symbol's body (no whole-file read)
vex show "PaymentService"

# Fuzzy / multi-word search (returns ranked neighbors when no symbol matches)
vex search "payment processing" --semantic

# Find all usages of a symbol (--strict drops string-literal / comment / wrong-scope noise)
vex usages "IndexReader" --strict

# File structure outline
vex outline src/main.rs

# Find implementations of a trait/interface
vex implementations "Iterator"

# Callgraph: who calls / is called by a function (fast path via persistent index)
vex callers "process_event"
vex callees "process_event"

# Multi-hop call graph (v1.7)
vex paths "main" "process_event"          # all caller chains from main → process_event
vex reachable "process_event"             # everything that transitively reaches it
vex tests-for "process_event"             # tests covering process_event (path globs + name heuristic; framework label per row)

# Symbol-level diff against a branch (v1.7)
vex diff --base main                      # what symbols did this branch change?

# Historical view of a symbol — every commit that touched it (v1.15.0; v1.16.0 expanded)
vex index --history                       # build the persistent history sidecar once
vex history "PaymentService"              # ~10ms — every version reachable from HEAD
vex history "PaymentService" --diff       # unified diffs between consecutive versions
vex history "Foo" --since 2026-01-01 --author alice --kind function
vex history "deleted_symbol" --exact-presence    # exact commit set where each blob lived (revert-aware)

# Semantic similarity by existing symbol — explain what's actually similar (v1.7)
vex similar "PaymentService" --limit 5 --min-score 0.7 --explain

# Near-duplicate pairs with reasoning (v1.7)
vex duplicates --min-score 0.95 --min-body-lines 5 --explain

# Search with per-call scope + metadata filters (v1.7)
vex search "Repository" --include 'src/**' --exclude '**/*.gen.*' --visibility public --async-only

# Why did the search return these results? (v1.7)
vex search "Foo" --why 2>trace.json

# Bundle: 4 round-trips → 1 envelope (v1.9, Phase 13.2)
vex bundle --mode symbol --symbol PaymentService          # body + callers + callees + similar
vex bundle --mode pr-impact --base origin/main            # changed symbols + transitive callers + tests
vex bundle --mode project --top-n 30                      # top-N by reverse call-graph indegree

# Diff-context filters on every search-shaped command (v1.9, Phase 13.7-D3)
vex search "Repository" --since-branched                  # only files changed since branching from main
vex usages "Config" --since HEAD~3                        # refs within the last 3 commits
vex callers "Foo" --changed-only                          # working-tree changes only

# Extract just a symbol's body — replaces Read for a specific function/class
vex show "PaymentService"                                 # full body of the class / fn
vex show "Foo" "Bar" "Baz"                                # multiple symbols in one call

# Smart show truncation for token efficiency (v1.9, Phase 13.3)
vex show "BigClass" --signature-only                      # just the signature line
vex show "PaymentService" --head 20                       # first 20 lines of the body
vex show "Foo" --no-body                                  # signature + docstring, no body

# Ranking-eval harness — CI regression guard (v1.9, Phase 13.12)
vex eval --bench benches/ranking_golden/queries.toml      # nDCG@10 / recall@10 / MRR per query
vex eval --min-ndcg 0.85                                  # fail if mean nDCG drops below threshold

# Capability discovery for MCP clients (v1.9, Phase 13.0)
vex capabilities                                          # JSON: protocol_version, signals, bundle_modes, …

# Fast existence check
vex check "Foo" "Bar" "Baz"

# Incremental update (re-parses only changed files, reuses unchanged from index)
vex update

# Watch mode (re-indexes on file changes)
vex watch

# Multi-repo: treat a set of sibling repos as one workspace (v1.22.0)
vex index --workspace                     # build every member of .vex-workspace.toml
vex search "RetryPolicy" --workspace      # fan out, results grouped by repo
vex usages Config --strict --workspace    # cross-repo strict refs (v7+ index)
vex watch --workspace                     # keep every member incrementally fresh

# Show index stats
vex status

# GPU doctor — is the compiled EP actually engaging on this machine? (v1.16.0)
vex gpu                                   # probes the compiled-in EP with strict registration
vex gpu cuda                              # narrow to one EP
vex gpu --enable                          # persist working device to VEX_DEVICE

# Shell completions
vex completions zsh > ~/.zfunc/_vex

Commands

Per-query filters (every search-shaped command)

All search-shaped commands accept scope filters. Specific filters vary by command:

• --include <glob> / --exclude <glob> (repeatable, gitignore syntax) — per-call path scoping that doesn't require re-indexing. --exclude wins over --include. Example: vex search Foo --include 'src/**' --exclude '**/*.gen.*'. --exclude-tests (MCP: exclude_tests: true) drops test files (tests/, *_test.*, test_*.py, *.spec.ts, __tests__/, tests.rs, … — the same set as vex tests-for); it composes with --include/--exclude, is recorded in the --why trace, and is applied by every command that takes the scope flags (search, usages, callers/callees, impact, grep, show, pattern, implementations/subtypes, similar/duplicates, modules, paths, reachable, diff, bundle pr-impact — there it filters the changed files and the transitive-caller/test rows; bundle symbol/project modes ignore all scope filters). vex tests-for rejects --exclude-tests (exit 2): it lists test functions, so the flag would always return nothing. Path-based only: Rust unit tests inside a #[cfg(test)] mod tests block of a non-test file are not excluded.
• --filter-path <substring> (alias --filter) — path-substring filter on search, show, usages, grep, similar, duplicates. Composes AND with the globs.

vex search / vex show additionally accept:

• --visibility <public|private|protected|internal> — keep only symbols whose signature carries the explicit keyword. Defaults aren't inferred (bare Rust fn foo() does NOT match --visibility private).
• --async-only / --no-async — keep or exclude async / Kotlin-suspend symbols.
• --static-only, --sealed-only — restrict to static class members or sealed (or Java-final) types.

Reasoning flags

• vex search --why prints a JSON trace to stderr (the result list stays on stdout): normalized_query, per-channel hit counts (FST / BM25 / semantic), fallbacks engaged (fuzzy), and the active filter snapshot.
• vex pattern --why prints a JSON ScanTrace to stderr after the result list: mode (indexed / live_scan), root_kind_inferred, candidate_files / total_files, and fallback_reason when the indexed prefilter was skipped (no-index, no-skeleton-section, empty-section, grammar-drift, partial-section, index-open-error). MCP callers see the same JSON under _meta.why.
• vex similar --explain / vex duplicates --explain add a jaccard overlap score plus a truncated unified diff between the two bodies, so you can decide whether two semantically-clustered symbols are actually duplicates before acting.

Multi-repo workspaces (--workspace) — v1.22.0

Declare a set of sibling repos in a .vex-workspace.toml and run any command with --workspace to fan out across all of them, grouped by repo:

# .vex-workspace.toml (at the directory that contains the repos)
members = ["./api", "./worker", "./shared-lib"]

vex index --workspace                   # build every member into its own per-repo index
vex update --workspace                  # incremental refresh, per-repo changed/deleted counts
vex usages Config --strict --workspace  # cross-repo strict refs, grouped by repo

• --workspace is accepted by index, update, search, grep, check, usages, impact, callers, callees, reachable, modules, and watch. Each member keeps its own .vex.toml (excludes / embedder / sections / cache).
• Reference and call-graph resolution is per-repo by default. The one exception is vex usages <name> --strict --workspace, which resolves a reference in repo B to a symbol defined in repo A via a gtags-style name fallback (rendered as a name-resolved sub-tier). Requires v7+ index — re-run vex index after upgrading.
• A member missing a capability (--strict on an old index, a call graph for reachable) is reported unavailable for that repo instead of aborting the whole fan-out. --workspace conflicts with --why. See docs/MULTIREPO.md and LIMITATIONS §7.

Configuration

Create a .vex.toml in your project root to customize vex behavior:

vex init  # generates .vex.toml with commented defaults

# .vex.toml

# Glob patterns to exclude from indexing (gitignore syntax, on top of .gitignore)
exclude = [
    "vendor/**",
    "node_modules/**",
    "*.generated.go",
]

# Output format — "compact" (default since v1.10.1; single-line records),
# "text" (verbose multi-line), or "json" (envelope for MCP / tools).
# format = "text"

# Enable semantic embeddings by default
semantic = true

# Automatically update index before search if stale
# auto_update = false

# With auto_update, refresh in the background rather than blocking the query:
# answers come from the index on disk, the response is flagged stale, and the
# rebuild lands for the next query. Trades freshness for latency.
# async_update = false

# GPU device for semantic indexing (GPU-enabled builds only). "auto" uses the
# compiled-in GPU EP when it initializes, else CPU; or "cpu"/"cuda"/"directml"/
# "coreml". `gpu = true/false` is shorthand for auto/cpu. See GPU Acceleration.
# device = "auto"
# gpu = true

# Embedder model: minilm-l6-v2 (default), jina-code, bge-base-en-v1.5,
# bge-large-en-v1.5, mxbai-large. Changing it requires a reindex.
# Set globally across projects with the VEX_EMBEDDER env var (this file wins).
# embedder = "minilm-l6-v2"

# VCS backend for diff-scoping (--since/--since-branched/--changed-only).
# "auto" (default) detects .git/.svn/.arc; "git" | "none" | "arc" | "svn".
# git, arc (Yandex Arc), and svn (Subversion) are all functional backends;
# svn declines --since-branched (no merge-base). "none" disables diff-scoping.
# Overridden by the --vcs flag and the $VEX_VCS env var. See docs/VCS-BACKENDS.md.
# vcs = "auto"

CLI flags always override config values. Use --no-semantic to explicitly disable semantic mode when the config enables it. The VEX_DEVICE and VEX_EMBEDDER environment variables act as global defaults across all projects (lowest precedence, below .vex.toml) — see GPU Acceleration.

Keeping config out of the repo

Don't want a .vex.toml inside the repository (can't .gitignore it, shared checkout, etc.)? vex never creates one on its own — only vex init writes it — and you have two ways to keep config external:

• --config <path> / $VEX_CONFIG — point vex at a config file anywhere on disk. It replaces the in-repo lookup entirely, so the repo stays clean:
  server.tsCopy

  vex --config ~/vex/this-repo.toml search Foo
  export VEX_CONFIG=~/vex/this-repo.toml   # or set it once per shell
  

  --config beats $VEX_CONFIG; a missing/invalid path is a hard error (vex won't silently fall back). Relative paths inside that file resolve against the file's own directory.
• A parent directory — config lookup walks up from the project to the filesystem root, so a .vex.toml placed in any ancestor (e.g. ~/work/.vex.toml, or ~/.vex.toml for a machine-wide default) is picked up for every repo beneath it, with none living in the repos themselves.

The index itself is never written into the repo — it lives in the cache dir (--cache-dir / $VEX_CACHE_DIR / platform cache), so a clean repo is just a matter of config placement.

Staleness Detection

Vex detects when the index is stale and warns before search:

$ vex search "Config"
Warning: index may be stale (HEAD changed). Run `vex update`.

How it works: on every search, vex compares the git HEAD stored at index time with the current HEAD (~0.1ms, single git rev-parse). If HEAD changed → stale. For non-git repos, falls back to mtime comparison — and since v1.11 (H11), when mtime fires, vex streams a xxh3_64 content hash of the file and compares it to the manifest. If the hash matches, the touch was cosmetic (git checkout, rustfmt no-op, rsync --times) and the file stays Fresh; only a real content change re-triggers indexing.

Auto-update: skip the warning and update inline:

# Per-command
vex search "Config" --auto-update

# Always (in .vex.toml)
auto_update = true

# Disable staleness check entirely
vex search "Config" --no-stale-check

GPU Acceleration

Semantic indexing (--semantic) can run the embedding model on a GPU — a large win on a full/cold index. On an RTX 3080 over a 28k-symbol C++ module, embedding the default MiniLM model was 51× faster on CUDA and 29× on DirectML vs CPU (full benchmark + design notes in docs/GPU_SUPPORT.md).

Two layers — the binary, and the device:

• Prebuilt binaries bake in a driver-only GPU EP: Windows → DirectML (any DX12 GPU — NVIDIA/AMD/Intel; the redist DirectML.dll is bundled in the archive), macOS arm64 → CoreML. No SDK, no extra install. The Linux prebuilt is CPU-only.
• CUDA is a source-build opt-in (fastest on NVIDIA — ~1.75× DirectML): cargo install --git https://github.com/tenatarika/vex vex-search --features gpu-cuda. Needs the CUDA 12 runtime + cuDNN 9 on PATH (the NVIDIA driver alone is not enough — it ships only nvcuda.dll, not the runtime/cuDNN). Source builds for the others: --features gpu-coreml / gpu-directml.

Selecting the device (vex index / vex update):

• --gpu / --no-gpu — force GPU (Auto) or CPU.
• --device cpu|auto|cuda|directml|coreml — pick a specific execution provider.
• Default is Auto: use the compiled-in EP when it initializes, else silently fall back to CPU. On a CUDA-enabled binary Auto prefers CUDA → DirectML → CoreML; on the standard Windows prebuilt only DirectML is compiled in, so Auto uses DirectML regardless of whether the PC could do CUDA.
• A tiny incremental vex update stays on CPU (the GPU warm-up isn't worth a handful of symbols); cold/large --semantic builds use the GPU.

Is the GPU actually being used? Run vex gpu — it reports the compiled EP and actively probes it, so a silent CPU fallback shows as FAILED with targeted setup remediation. vex gpu cuda probes a single EP; vex gpu --enable persists the working device to VEX_DEVICE (user env via setx on Windows; prints the export line to add on macOS/Linux). A stale VEX_DEVICE pinned to a GPU EP that a later (e.g. CPU-only) build lacks degrades to CPU rather than erroring.

Environment variables

Output Formats

# Compact single-line records — default since v1.10.1 (token-efficient, agent-friendly)
vex search "Foo"

# Verbose multi-line / human-readable
vex search "Foo" --format text

# JSON envelope (for MCP / tool integration; what `vex-mcp` parses)
vex search "Foo" --format json

Pin a different default in .vex.toml via format = "text" if you want the verbose multi-line view at the terminal.

JSON envelope (v1.11.0 — BREAKING for bare-array parsers)

Every --format json subcommand wraps its payload in the Phase 13 envelope. Single shape, easy to detect via protocol_version:

{
  "protocol_version": "v1",
  "capabilities": { /* see `vex capabilities` */ },
  "_meta": { "vex.dev/index_age_ms": 1200, "ttlMs": 30000, "cacheScope": "project" },
  "results": [ /* the actual data, shape depends on the subcommand */ ]
}

Pre-v1.11 only search and bundle returned this envelope; the other subcommands (show, usages, pattern, grep, implementations, callers, callees, paths, reachable, tests-for, check, similar, duplicates, diff, outline, index, update, status, eval) emitted bare arrays / objects. Migration: pre-1.11 jq '.[0].name' or data[0]['name'] now needs jq '.results[0].name' / data['results'][0]['name']. Detect the envelope via response.get('protocol_version') == 'v1' to support both shapes during a rollout window.

How Search Works

Structural Search (default)

Searches by symbol name using an inverted index with CamelCase splitting:

• "PaymentService" — exact match
• "Payment" — prefix match, finds PaymentService, PaymentGateway
• "payment" — case-insensitive, also finds via CamelCase tokens

Semantic Search (--semantic)

Embeds your query with MiniLM-L6-v2 (384-dim vectors) and finds symbols with similar meaning:

• "parse source code files" finds parse_file, extract_refs, parse_file_symbols
• "database storage" finds populate_db, create_10k_db, add_root_persists_to_db
• "find implementations of an interface" finds find_implementations, test_interface_extends

BM25 Channel (auto-on when index has BM25 data)

A classic Okapi BM25 (K1=1.2, B=0.75) over symbol body tokens — identifiers, signatures, docstrings. Closes the gap between "exact name" (structural) and "general meaning" (semantic): finds rare body terms like timeout, retry, singlestore, idempotency_key that aren't part of any symbol name. Since v1.11 (Phase 8.4) body tokens are also extracted from TOML / YAML / HTML / CSS values, so vex search "production endpoint" --semantic can hit a [server] table with endpoint = "https://...". Pass --no-bm25 to disable per-call.

Hybrid Search (3-way RRF)

When the index has all three channels (built with --semantic), vex search fuses structural + BM25 + semantic using Reciprocal Rank Fusion. Symbols hit by ≥2 channels rank as Hybrid; symbols unique to one keep their original match type. Cuts both structural-noise and semantic-blur in the same query.

Usages (FST)

References stored in an FST (Finite State Transducer) — zero-copy lookup from mmap with prefix search support.

Symbol clusters (vex modules)

vex index groups symbols into clusters of code that call or reference each other (deterministic Leiden-CPM over the call, reference and hierarchy edges; no randomness, so two indexes of the same tree agree). vex modules reads them back:

vex modules                         # clusters of >= 3 symbols, largest first
vex modules --members 5 --sort cohesion --include 'src/**'
vex modules IndexReader             # the cluster of one symbol, with its members
vex modules --format text           # text output (shown below)

Modules — leiden-cpm/1 γ=1/8 · 475 clusters (≥3, showing 3) · 1,469 unclustered · 1,464 not eligible
  #60  src/cli/                     39 symbols  cohesion 0.49  hubs: OutputFormat, print_envelope, default_meta_for
  #23  crates/vex-mcp/src/tools/    38 symbols  cohesion 0.68  hubs: opt_bool, build_command, opt_u64
  #286 src/pattern/matcher/tests.rs 32 symbols  cohesion 0.78  hubs: parse_pattern, find_matches, Segment

(Output above is from this repository at the time of writing; cluster ids are ordinals in the section, not ranks.)

• A cluster's label is the deepest path prefix holding at least 60 % of its members' files. cohesion is internal / (internal + cut) edge weight; the hubs are the three members best connected inside the cluster.
• --include/--exclude/--exclude-tests filter members and hubs (out-of-scope hubs are dropped); a cluster is shown when at least one member is in scope, and its size is the in-scope count (size_at_build keeps the build-time count in JSON).
• Symbol mode reports a per-match status: clustered, unclustered (isolated), not_eligible (headings, modules, markup/config languages) or new_since_build.
• Clusters are computed by vex index and carried, frozen, across vex update. After an update the JSON has stale: true and new_since_build, and text output ends with a ! line; run vex index to recompute. This is separate from _meta.vex.dev/stale, which still means "index older than the working tree".
• --limit must be at least 1; with a SYMBOL it caps the matched symbols (JSON symbols_total reports the uncapped count) and --min-size is ignored. Exit codes: 0 with results, 1 when empty (the reason is in results.empty_reason and on stderr), 2 for a corrupt cluster section. With --workspace, clusters are per repo and --limit applies per repo.

Type-aware refs (--strict)

vex usages --strict <name> reads the v5 reference_edges section written by an LSP-style scope binder. For the languages with a binder (Rust, TypeScript, Python, C#, C++, Go, Java, Kotlin) every ref is resolved at index time against an in-file scope chain plus an import/use graph, then serialised against the global symbol the user actually meant — not just any line that mentions the spelling.

What this changes for the user:

• Identifiers inside comments, doc-strings, string literals, and regex bodies are dropped (this filter is on for everyone, not just --strict).
• A name shadowed by a let / const / fn param resolves to the inner scope, not the outer.
• A use ext::Foo; / import { Foo } from './ext' / from ext import Foo makes a ref to Foo resolve cross-file to whatever defines it in the index. For C++, quoted #include "..." (v1.14+) walks the transitive include graph via BFS to resolve Foo against symbols defined in any reachable header. System headers <vector> / <string> and macro includes (#include MY_HEADER) stay unresolved by design.
• A name imported but never defined in the index stays Unresolved and produces no edge — better than a coincidental match.

Without --strict vex usages still works for every supported language via the legacy refs FST; --strict simply trades recall breadth for precision on the eight binder languages. v3 / v4 indexes predating the binder bail with a "re-run vex index" message.

Structural Patterns (vex pattern)

Match code by shape rather than text. Works on every language vex parses; an indexed prefilter (via the v6 pattern_skeletons section) speeds up candidate selection on the 15 languages that emit skeletons (every language except Bash, Lua, YAML, and TOML, which live-scan).

Syntax:

• $NAME — capture a single identifier or balanced expression. Same name appearing twice enforces a back-reference: record($X, $X) matches record(state, state) and rejects record(state, other).
• $_ — wildcard (matches without capturing).
• $$$ — anonymous ellipsis (matches anything up to the next literal; spans newlines).
• $$$BODY / $$ARGS — named multi-line ellipsis. Functionally identical to $$$ but captures the consumed text under the given name; $$$BODY reads naturally for block bodies, $$ARGS for parameter lists. Back-reference equality also applies.
• && (space-flanked) — AND composition. Both sub-patterns must match in the same file, and shared metavar names must capture the same text in both: struct $S && impl $S matches files that have both shapes for the same $S.
• || (space-flanked) — OR composition (union, deduped by (path, line)). && binds tighter than ||.
• Composition operators only fire at bracket / quote depth 0, so record($X, $X) and f($X && $Y) stay single patterns.

Indexed prefilter: when a v6 index is present, the leading literal keyword of the pattern (fn, struct, class, def, impl, …) is mapped to a tree-sitter node kind, and vex pattern walks only the files whose persisted skeletons contain that kind. Visibility / async / export modifiers in front of the keyword are stripped before the match (pub async fn $F infers function_item correctly). Falls back to live-scan on grammar drift, missing section, or a partial section after vex update — --why reports the exact reason.

Examples:

# Multi-line function body with named captures
vex pattern 'fn $NAME($$ARGS) -> Result<$T, $E> { $$$BODY }' --lang rust

# Both struct and impl for the same type in one file
vex pattern 'struct $S && impl $S' --lang rust

# Interface OR class with the same name
vex pattern 'interface $N || class $N' --lang typescript

# See which mode and what narrowing happened
vex pattern 'fn $N($$$)' --lang rust --why 2>trace.json

Benchmarks

Compared against ast-index v3.31.0 (SQLite + FTS5) and ripgrep 15.1.0.

Methodology: indexing re-measured 2026-08-12 on Apple Silicon (macOS), vex v1.25.5, release build, cold cache, non-semantic index. Search figures are unchanged from the 2026-07-11 / v1.25.1 run (nothing in v1.25.2-v1.25.5 touches the search path) and are the average of 10 runs. Reproduce with ./benches/bench.sh (point VEX_BENCH_LARGE_PROJECTS at your own repos for larger corpora). Numbers are machine-specific — treat the ratios, not the absolutes, as the signal.

Indexing

Honest read: vex indexing is ~1.7-1.9x slower than ast-index — it builds far more at index time (FST + BM25 + persistent call graph + resolved reference edges + a v8 type-hierarchy section + a trigram skip-index + pattern skeletons + symbol clusters (v9)), where ast-index builds a SQLite + FTS5 store. That one-time cost buys the constant-time queries below; the resulting index is still ~1.4-2x smaller on disk (mmap + FST vs SQLite). These measurements are from v1.25.5 (before v9); v9 adds the cluster pass — re-measure pending. The gap was ~2.5-3x when last measured at v1.25.1; v1.25.5 is the main reason it has narrowed — a single-variable A/B against v1.25.4 put its cold-index gain at −34% and −31% on two corpora, from parsing each file once and sharing the tree across all extractors instead of re-parsing it per extractor. Projects indexed with --semantic are slower again (ONNX embedding generation) and produce a larger index.

Search: vex vs ast-index vs ripgrep

Medium project (ast-index repo, 31K lines Rust, avg 10 runs):

Key takeaway: vex search is constant ~4-5 ms (FST O(query_len)) regardless of project size — this is the win the slower index build pays for. The ripgrep comparison is not apples-to-apples: rg scans raw text with no index, so it scales with corpus size (single-digit ms on this 31K-line repo, 100 ms+ on large ones), while vex does a pre-built FST lookup. The durable advantage is amortized and qualitative: vex returns only symbol definitions (precise, token-efficient), while rg returns every text occurrence (noisy, expensive in LLM contexts).

Pattern Matching (vex only)

Medium project (ast-index repo, Rust):

ast-index and ripgrep do not support AST pattern matching.

Semantic Search

Illustrative (semantic capability, not a latency benchmark) — queries where structural search returns 0 results but semantic finds relevant symbols:

HNSW vs Brute-Force (semantic vector search)

The latency figures below were measured on an earlier build and not re-run in the v1.25.1 pass — read them as the scaling shape (HNSW stays flat, brute-force grows linearly), not current absolutes.

Semantic search embeds the query via ONNX (~55ms) then searches stored vectors. HNSW (usearch) replaces brute-force O(N) scan with O(log N) approximate nearest neighbor search:

HNSW stays constant ~3ms regardless of index size. Brute-force grows linearly. Total semantic search latency is dominated by ONNX embedding (~55ms), so end-to-end speedup is modest for small codebases but critical at scale.

LLM Token Efficiency

When an AI agent searches code, the output goes directly into the context window. Grep-based tools return every text occurrence — including comments, strings, variable usage, and matches in minified files — consuming tokens without adding signal.

vex returns only symbol definitions in a compact one-line format, drastically reducing token consumption:

Example — searching for a class name on a large project:

# rg: 20 matches across imports, usage sites, comments, tests (2,045 chars)
$ rg -w "PreAggregatedConfig" .
./models.py:3602:class PreAggregatedConfig(models.Model):
./models.py:3610:    pre_aggregated_config = PreAggregatedConfig.objects.get(...)
./serializers.py:48:from .models import PreAggregatedConfig
./tests.py:12:    config = PreAggregatedConfig(...)
... (16 more lines)

# vex: 1 definition (93 chars)
$ vex search "PreAggregatedConfig" --format compact
C PreAggregatedConfig models.py:3602 class PreAggregatedConfig(models.Model):

For an agent making 10-20 code lookups per task, vex saves 5,000-20,000 tokens per session compared to grep — reducing cost and leaving more context window for reasoning.

Supported Languages

19 languages indexed via tree-sitter. The capability columns:

• Binder — does vex usages --strict resolve refs through an LSP-style scope chain (Phase 11.1)? cross-file includes use / import resolution; in-file resolves within a file but treats imports as unresolved. The remaining languages fall back to the line-based scanner used by plain vex usages.
• Patterns — does vex pattern get the v6 indexed prefilter (Phase 11.4)? indexed means a persisted skeleton section narrows candidate files at query time; live-scan means tree-sitter walks every lang-matching file on each query. All 19 languages work with vex pattern syntax ($NAME, $$$BODY, && / ||); the prefilter just speeds up discovery for the 15 languages that emit a skeleton section (every language except Bash, Lua, YAML, and TOML).

See docs/SUPPORTED_LANGUAGES.md for grammar versions, ABI level, and the runbook for adding a language or upgrading a grammar. Adding a language to the indexed-Patterns tier is one allowlist edit in src/pattern/skeleton/kinds.rs — the Phase 11.4 follow-up promotion (Go → Java → Kotlin → C# → C++ → Swift → PHP → Ruby, plus SQL / Markdown / CSS / HTML) is complete; only Bash, Lua, YAML, and TOML remain on live-scan.

Index Location

macOS:     ~/Library/Caches/vex/<hash>/index.vex
Linux:     $XDG_CACHE_HOME/vex/<hash>/index.vex (fallback: ~/.cache/vex/<hash>/index.vex)
Windows:   %LOCALAPPDATA%\vex\<hash>\index.vex   (fallback: %USERPROFILE%\AppData\Local\vex\<hash>\index.vex)

Each project gets its own index based on a hash of the canonical project root path (xxh3). Overrides:

• --cache-dir <path> — point vex at a custom cache directory
• $VEX_CACHE_DIR — environment variable (lower precedence than --cache-dir)
• cache_dir in .vex.toml — configuration file (lowest precedence)

Known limitations

vex is a static-analysis tool — some real call sites and references are invisible by construction. The headline gaps:

• vex callers outside function scope — Module-level expressions are reported via synthetic <module:path> callers (Phase 14.1). Python + Java function/method decorators (Phase 14.2), Kotlin annotations + C# method/constructor attributes (Phase 14.2.2), and TypeScript method decorators + Rust outer attributes on fns/methods (Phase 14.2.1) emit forward edges. Remaining gap: class-level decorators (Phase 14.6); Rust #[derive(...)] is intentionally filtered.
• vex usages quality depends on language. Rust / TypeScript / Python / C# / C++ get --strict (binder-resolved refs from the v5 reference_edges section, Phase 11.1). Other languages use a line-based identifier scan with a higher false-positive rate.
• Dynamic dispatch is invisible. String-resolved factories (uvicorn.run("main:app")), task queues (celery_task.delay()), reflection (getattr(obj, name)()) — none of these produce edges.
• Workaround: vex grep '\bname\b' is the exhaustive textual fallback. Slower (~50 ms) but never misses a hit.

See docs/LIMITATIONS.md for the full coverage matrix, repros, and recommendations per query type.

Troubleshooting

Surfacing internal warnings

Vex emits structured logs via the tracing crate at parse/store boundaries — failed grammar loads, mmap reopens, manifest mismatches, and so on. By default RUST_LOG is unset, so only the most critical diagnostics make it to stderr.

When a search returns surprising results or an index command behaves oddly, raise the log level:

RUST_LOG=vex=warn vex search Foo
RUST_LOG=vex=info vex index   # noisier — file-level progress

For what the search engine actually did (per-channel hit counts, fuzzy fallback engagement, applied filters), use the structured trace instead:

vex search Foo --why 2>trace.json   # trace lands on stderr as JSON

See docs/MCP-SCHEMA.md for the --why / why: true JSON shape.

Integration

Claude Code (CLI Integration)

The recommended way to integrate vex with Claude Code is via CLAUDE.md rules (see below). Vex runs as a CLI tool — Claude Code calls it directly via Bash, no MCP server needed.

Setup:

# Install vex
brew tap tenatarika/tap && brew install vex

# In your project
cd /path/to/project
vex init              # create .vex.toml
vex index             # build index (add --semantic for meaning-based search;
                      # add --history for `vex history <Symbol>` archaeology queries — v1.15.0/v1.16.0)

Then add .vex.toml config for auto-update so Claude always searches a fresh index:

# .vex.toml
auto_update = true
# format = "compact"   # already the default since v1.10.1 — set "text" if you'd rather see verbose output

Multi-repo (v1.22.0): if Claude Code is working across several repos at once, drop a .vex-workspace.toml at the common parent and tell Claude to add --workspace to its vex calls — e.g. vex usages Config --strict --workspace to trace a symbol's references across every repo, or vex check Foo --workspace to see which repos define it. Results come back grouped by repo. See Multi-repo workspaces.

Claude Code (MCP Server)

Alternatively, vex includes an MCP server (vex-mcp) that exposes all commands as MCP tools. Note: Homebrew installs only vex (not vex-mcp). Since v1.11.2 a prebuilt vex-mcp binary ships in every release alongside vex for the three triples the build matrix covers: aarch64-apple-darwin (macOS Apple Silicon), x86_64-unknown-linux-gnu (Linux), and x86_64-pc-windows-msvc (Windows). Intel-Mac and other triples still require the source build below.

Easiest setup (v1.15.0+):

vex mcp install --agent claude-code

This runs claude mcp add --scope user --transport stdio vex --env VEX_ROOT=<root> -- <vex-mcp> for you (v1.27.1+). If the claude CLI is not on PATH, it prints that command instead and writes nothing. Releases before v1.27.1 wrote ~/.claude/claude_desktop_config.json, which Claude Code does not read; re-run the command after upgrading.

Manual setup:

# 1. Download the prebuilt for your platform from
#    https://github.com/tenatarika/vex/releases/latest
#    e.g. vex-mcp-aarch64-apple-darwin.tar.gz / vex-mcp-x86_64-pc-windows-msvc.tar.gz
# 2. Extract and put the binary on PATH (or remember the full path).

# Source build (if you prefer or are on an unsupported triple)
cargo build --release -p vex-search-mcp

# Register with Claude Code (user scope; Claude Code keeps it in ~/.claude.json)
claude mcp add --scope user --transport stdio vex \
  --env VEX_ROOT=/path/to/your/project --env VEX_DEVICE=auto \
  -- /path/to/vex-mcp

# Or project scope: commit a .mcp.json at the project root
# (see integrations/claude-code/mcp.json)

VEX_DEVICE (v1.16.0) picks the GPU execution provider when the binary was built with gpu-cuda / gpu-directml / gpu-coreml — relevant when an MCP-driven index / update call rebuilds semantic embeddings on a large repo (51× CUDA / 29× DirectML over CPU on MiniLM-L6). auto is safe on CPU-only builds (degrades silently). Run vex gpu once to confirm the EP actually engages.

MCP Tools (28):

• search — 3-way hybrid (structural + BM25 + semantic); accepts filter / include / exclude / kind / context_path / no_bm25 / --why / metadata filters / diff-scope (since / since_branched / changed_only)
• find_symbol — exact name lookup
• find_similar — semantic search by free-form description
• similar — nearest neighbors of an existing symbol (explain adds Jaccard + diff); diff-scope
• duplicates — near-duplicate symbol pairs (explain shows what differs); diff-scope
• show — extract symbol body from source; Phase 13.3 truncation flags (signature_only / head / no_body / collapsed, mutually exclusive)
• outline — file structure
• usages — find all references to a symbol; filter_path / strict / why
• impact — delete-safety blast radius (verdict + per-channel evidence); depth / exclude_docs
• grep — regex content search
• pattern — AST pattern matching with metavar back-references; diff-scope; --why
• implementations — find types extending a base class/trait/interface (incl. generics); diff-scope
• subtypes — transitive-down closure over extends/implements edges (direct children, grandchildren, …), depth-labelled; index-only (no live-walk fallback); depth / diff-scope
• modules — de-facto modules: clusters of symbols that call/reference each other (v9 index, computed on full vex index); list clusters (label, size, cohesion, hubs) or pass symbol for its cluster; limit / min_size / members / sort / scope / workspace; empty result + empty_reason on older indexes or --no-clusters
• callers / callees — direct callgraph navigation (fast path via persistent index); diff-scope
• paths — enumerate caller chains between two functions
• — transitive callers of a target

Note: vex history and vex tests-for were promoted to first-class MCP tools in v1.20.0 (D5); earlier docs that called history "CLI-only" are stale. Both emit the same --format json envelope as every other vex command.

Multi-repo (v1.22.0): eleven tools — search, grep, check, usages, impact, callers, callees, reachable, modules, index, update — take a workspace: boolean arg that fans the call across every .vex-workspace.toml member, returning the grouped {workspace, repos:[...]} payload under structuredContent.results. Point project_root at or above the .vex-workspace.toml. find_symbol is excluded (use check/search); why is ignored in workspace mode. See docs/MULTIREPO-PHASE8-mcp.md.

MCP ↔ CLI parity (v1.10): the schemas now mirror the CLI surface for every path-aware tool. Glob filters (include / exclude), substring filter, kind boost, context_path proximity hint, no_bm25, Phase 13.3 truncation, diff-scope, and no_stale_check are exposed everywhere the CLI accepts them — agents no longer need to drop to bash for "Rust files under crates/api/ since main"-style scoping.

The schemas follow a canonical vocabulary (query / symbol / symbols / path / pattern / filter / include / exclude); pre-v1.7 aliases (name, file, names, etc.) still work and emit _meta.deprecated_args: [...] in the JSON-RPC response. Malformed JSON-RPC input now returns the spec-compliant -32700 Parse error response (v1.9.2 fix) with a 512-codepoint echo of the offending line in the data field; broken-pipe / EOF on stdin cleanly shuts down the server instead of dropping in-flight tool calls. See docs/MCP-SCHEMA.md.

For other MCP-compatible clients (Cursor, Codex CLI, Windsurf, Cline, Continue.dev, Zed), see Other MCP Clients below — same vex-mcp binary, different config files.

Other MCP Clients

The same vex-mcp binary works with any MCP-compatible client. The binary install is identical to the Claude Code section above; only the per-client config file location and format differ.

One-line setup (v1.15.0+):

vex mcp install --agent cursor       # or any of: claude-code, codex-cli, windsurf, cline, continue, zed
vex mcp install --agent all          # fan out across every supported agent
vex mcp install --agent cursor --dry-run   # preview the post-merge config without writing

For file-based agents, vex mcp install reads your existing agent config, merges a single vex server entry without disturbing siblings, and writes back atomically. For Claude Code it runs claude mcp add instead of editing a file. Idempotent — re-running on a matching entry is a no-op skip (--force overrides). vex mcp uninstall --agent <X> removes the entry; vex mcp list enumerates current entries per agent. The config files documented below for the other agents are exactly what vex mcp install writes — keep integrations/ handy for manual edits, agents the auto-installer doesn't know yet, or anything more exotic than the default shape.

Copy-pasteable snippets for the most common ones live under integrations/:

Per-agent caveats (auto-approve flags, timeout overrides, agent-mode requirements) are documented in integrations/README.md.

MCP Registry (from v1.27.2): vex is listed in the official MCP Registry as io.github.tenatarika/vex. Each release attaches one MCP Bundle per platform (vex-mcp-<target>.mcpb, macOS arm64 / Linux x86_64 / Windows x86_64) that holds both vex-mcp and vex; an MCPB-capable client asks for the project root once and needs nothing else on PATH. Bundle installs are updated by the client, not by vex self-update (which refuses to run inside a bundle).

Glama: also listed in the Glama MCP directory, which builds the server and grades its tools.

Agent Recipes & Workflows

Once vex-mcp is wired into your agent, the next question is what to ask the agent so it picks the right tools in the right order. docs/COOKBOOK.md is a recipe collection for the common chains — code archaeology, cross-file refactor with usages --strict verification, PR-impact analysis via bundle(mode="pr-impact"), dead-code & duplicate cleanup, and multi-repo orchestration. Each recipe shows the tool sequence, the why of the ordering, and a phrase that reliably triggers the chain in agent prompts.

Documentation & Integration:

• Full vex documentation and API reference: https://context7.com/tenatarika/vex (on Context7)
• Agent skill reference: .claude/skills/vex/SKILL.md (included in the project)

Shell Integration

# Shell completions (tab-completion for commands and flags)
vex completions bash > ~/.bash_completion.d/vex   # Bash
vex completions zsh > ~/.zfunc/_vex               # Zsh (add ~/.zfunc to fpath)
vex completions fish > ~/.config/fish/completions/vex.fish  # Fish

# Aliases — add to .zshrc / .bashrc
alias vx="vex search"
alias vxu="vex usages"
alias vxi="vex index --path ."
alias vxs="vex index --path . --semantic"
alias vxw="vex watch"

CLAUDE.md Integration

Add this to your project's CLAUDE.md to make Claude Code use vex instead of grep:

## Code Search

Before first use in a project, run `vex init` to generate `.vex.toml`, then `vex index` to build the index.
Set `auto_update = true` in `.vex.toml` so the index stays fresh automatically.

Use vex for code search instead of grep or manual file reading:

- `vex check "SymbolName"` — exact-name lookup: does it exist? (~4ms)
- `vex search "SymbolName"` — fuzzy symbol search: find definitions by name or meaning
- `vex search "description" --semantic` — search by meaning (requires --semantic index)
- `vex search "rare_term"` — BM25 channel finds rare terms in symbol bodies (auto-on when index has BM25 data)
- `vex show "SymbolName"` — extract symbol body (use INSTEAD of Read for specific symbols)
- `vex show "A" "B" "C"` — extract multiple symbols at once
- `vex usages "SymbolName"` — find all references
- `vex usages "SymbolName" --strict` — refactor-grade refs (binder-resolved, high precision)
- `vex impact "SymbolName"` — delete-safety blast-radius report (safe/unsafe/uncertain)
- `vex modules [SYMBOL]` — de-facto code clusters (symbol communities)
- `vex pattern 'class $NAME(BaseModel):' --lang python` — AST pattern matching with metavariables
- `vex pattern 'fn $N($$ARGS) -> Result<$T, $E> { $$$BODY }' --lang rust` — multi-line `$$$BODY` / `$$ARGS` capture
- `vex pattern 'struct $S && impl $S' --lang rust` — AND composition (back-ref `$S` must agree across both shapes)
- `vex pattern 'interface $N || class $N' --lang typescript` — OR composition (union, deduped by `(path, line)`)
- `vex pattern '<pat>' --lang <lang> --why` — emit ScanTrace on stderr (mode / candidate vs total / fallback reason)
- `vex outline path/to/file.py` — file structure overview
- `vex implementations "BaseService"` — find types extending a class/interface
- `vex subtypes "BaseService"` — transitive-down closure over extends/implements edges (direct children, grandchildren, …)
- `vex callers "function_name"` — find all callers (~4ms via persistent call graph)
- `vex callees "function_name"` — find all callees (~4ms via persistent call graph)
- `vex paths "from" "to"` — enumerate caller chains between two functions (multi-hop)
- `vex reachable "Target"` — transitive callers of a target (blast-radius analysis)
- `vex tests-for "SymbolName"` — test functions that cover a symbol (framework-labeled)
- `vex history "SymbolName"` — historical versions of a symbol across commits
- `vex similar "SymbolName"` — semantically close symbols (requires --semantic index)
- `vex duplicates --threshold 0.95` — near-duplicate symbol pairs
- `vex diff --base main` — symbol-level diff against a branch (added / removed / moved / body-changed)
- `vex bundle --mode symbol --symbol Foo` — single-call body + callers + callees + similar (replaces 4 round-trips)
- `vex bundle --mode pr-impact --base origin/main` — changed symbols + transitive callers + tests on the current branch

Many search-shaped commands support `--filter-path "path/"` (alias `--filter`) to narrow results to a directory (e.g. `search`, `show`, `usages`, `grep`, `similar`, `duplicates`). Most search-shaped commands also accept `--since <rev>` / `--since-branched` / `--changed-only` for diff-scoping.

### Rules
- **Always prefer `vex show` over `Read`** when you need a specific function or class
- **Always prefer `vex search` over `Grep`** when looking for symbol definitions
- **Use `vex grep` instead of `Grep`** for searching inside string literals, comments, or config values
- **Use `--format compact`** for token-efficient output in automated workflows
- **Use `--kind fn`** to boost results matching a specific symbol kind (fn, struct, trait, class, etc.)
- **Use `--context-path`** with the path of the file you are currently editing to boost nearby results
- **Run `vex update` after modifying source files** if `auto_update` is not enabled in `.vex.toml`
- **Use `vex pattern ... --why`** to debug match counts — the trace tells you whether the indexed prefilter ran or fell back to live-scan, and why
- **Indexed pattern prefilter requires a full `vex index`** — after `vex update` the section is partial and `vex pattern` automatically degrades to live-scan (reason `partial-section` in `--why`)

### Indexing
- `vex index` — full structural index + pattern skeleton section (v6)
- `vex index --semantic` — with embeddings (slower, enables semantic search)
- `vex update` — incremental update (only changed files)
- `vex index --no-pattern-index` — skip the v6 pattern skeleton section if you don't use `vex pattern` (sticky across `vex update`)
- `vex index --no-clusters` — skip computing symbol clusters (v9). `vex update` keeps the opt-out, but the next plain `vex index` computes clusters again

Testing

Unit & Integration Tests

cargo nextest run --workspace          # ~4,000 tests — unit, integration, property-based, adversarial
cargo test --doc                       # doctests
cargo clippy -- -D warnings            # zero warnings policy

(nextest ≥ 0.9.145 is recommended — older versions report spurious LEAKs on macOS; update with cargo nextest self update.)

Test coverage includes:

• Per-language grammar regression (NEW): tests/<lang>_query_test.rs for all 19 supported languages — catches ABI mismatches and AST node renames when a tree-sitter grammar crate is upgraded
• Binary format: roundtrip, corrupted/truncated/wrong-version rejection, out-of-bounds access, string pool dedup, empty index
• Adversarial format: 20 crafted index tests — overflow offsets, bad magic/version, alignment attacks, truncated records
• Vectors: write/read roundtrip for 384-dim f32 embeddings
• FST: refs FST roundtrip, prefix search, symbol FST exact/prefix/fuzzy search
• Search: structural, fuzzy (Levenshtein), RRF fusion, reranking with kind/path/proximity boosts
• Reranking stress: NaN/Infinity/zero scores, 10K results, edge context paths
• Property-based (proptest): rerank preserves length, sorted output, no NaN/negative scores, fusion commutativity
• Incremental update: unchanged reuse, deleted removal, file rename, symbol move between files, empty file
• Concurrency: parallel index/update (lock serialization), concurrent readers, read during reindex
• Multi-language: Rust, Python, Go, Kotlin, TypeScript, C++, cross-language same-name, wrong extension, 1K-symbol file, deep nesting, error recovery
• Unicode: BOM, mixed CRLF, unicode identifiers, null bytes, empty/whitespace files
• Path edges: spaces in paths, deep nesting (20 levels), symlinks, absolute vs relative, Windows backslashes
• Callgraph: callers/callees for Rust, Python, Go, TypeScript, Java
• Persistent call graph (v1.5): format v4 roundtrip, callers/callees FST lookup, dedup, same-name-across-files isolation, same-name-within-file disambiguation, incremental update preserves edges, fallback to live scan for v3
• Similar/duplicates (v1.5): self-exclusion, threshold filtering, canonical pair dedup, body-length filter, empty-index handling
• Pluggable embedder (v1.5): registry lookup, mismatch detection (incl. back-compat for pre-9.1 manifests), config + CLI priority, writer variable vector_dim
• BM25 channel (v1.5): writer/reader roundtrip, pipeline emission, IDF discrimination, short-doc preference, 3-way RRF with Hybrid labeling, MatchType tagging, unicode tokens
• Staleness: git HEAD comparison, dirty file detection, mtime fallback

Fuzz Testing

Fuzz tests exercise every parser that consumes untrusted input — the binary index format, sidecar files, the user-facing pattern grammar, and the JSON manifest — using cargo-fuzz (libFuzzer + AddressSanitizer):

# Install (once)
cargo install cargo-fuzz

# Generate seed corpus for every target
bash fuzz/generate_seeds.sh

# Run (requires nightly)
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_index_reader        -- -max_total_time=120
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_refs_fst            -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_symbol_fst          -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_bloom_load          -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_pattern_parser      -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_manifest_load       -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_marker_load         -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_tokenize_document   -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_hash_index_load     -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_incremental_hnsw    -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_rename_chains_load  -- -max_total_time=60
RUSTUP_TOOLCHAIN=nightly cargo fuzz run fuzz_state_load          -- -max_total_time=60

Eighteen fuzz targets cover the reader's unsafe paths plus every text / sidecar parser that takes adversarial input:

Most recent system-wide audit (Q4-A/B closure, 2026-06-17): ~76M total executions across all 11 targets that existed at the time, 0 crashes / panics / AddressSanitizer hits / leaks (61s per target, libFuzzer + ASan + nightly). One latent FST panic was caught en route in a dead-code path on adversarial ref_edge bytes and fixed before the clean run — find_ref_edges_by_symbol and RefReader::find_by_prefix now wrap the inner FST walk in catch_unwind so a corrupt sidecar returns Err instead of taking down the process. Earlier baselines: v1.14.1 (2026-06-05) ran 5.8M iterations across 9 targets clean; v1.15.2 release-gate (2026-06-08) ran ~853k focused executions on the four highest-signal targets clean.

Fuzzing has found and fixed eight real defects across the project life:

• v1.x: out-of-bounds read on crafted symbol_count, misaligned pointer dereference on odd symbols_offset, unchecked section offsets exceeding file size (binary reader hardening).
• v1.12.0: SymbolBloom::load accepted a sidecar with n_bits = 0
  • k_num = 0 whose consistency guard passed but later panicked inside bloomfilter::Bloom::check on hash % 0. Fix: reject degenerate sizes during load.
• v1.12.0: SymbolBloom::load accepted k_num up to ~2.1B, which made every may_contain call loop for 110+ seconds (DoS, not a panic). Fix: cap k_num <= MAX_K_NUM = 64 at load time.
• Phase 11.1.10 / Q4-A (2026-06-17): FST walk inside find_ref_edges_by_symbol panicked on a crafted refs FST, bypassing the production catch_unwind (the libfuzzer-sys panic hook fires before user code can intercept). Fix: wrap the inner walk in its own catch_unwind and surface Err. Defense-in-depth hardening was also applied to Manifest::load: a 128 MiB pre-read size cap blocks hostile JSON before serde can allocate multi-GB heap (Q4-B audit follow-up; defense-in-depth, threat model is user-owned files).
• v1.23.0: a 451-byte malformed Kotlin input drove tree-sitter's GLR error recovery into super-linear time and memory (334 s, >2 GB; DoS, not a crash; found by fuzz_kotlin_binder). Fix: every production parse goes through parser_pool::parse_text, which caps progress-callback invocations (scaled by input size) for all languages.
• v1.23.0: tree_sitter::Node::utf8_text() panicked on malformed input where tree-sitter emitted a node past EOF (found by fuzz_kotlin_binder). Fix: the bounds-checked NodeTextExt (node_text / node_text_opt) replaces raw utf8_text in the extractor, binders and pattern prefilter.

The v1.13.0 / v1.14.1 additions found no defects in fresh code — the review-driven MAX_COUNT guards on hash_index::save / load were added as defence-in-depth before the fuzzer ran (rust-reviewer + code-reviewer flagged the truncating as u32 cast on save), and the sustained 3M / 5.8M iteration runs confirmed they hold.

Architecture

CLI (clap) → Pipeline (rayon, 500-file chunks) → Tree-sitter
                                      ↓
                           Binary format v9 (mmap, zero-copy)
                                      ↓
       ┌──────────────────┬──────────────┬──────────────┬──────────────┬─────────────┬──────────────┐
       ↓                  ↓              ↓              ↓              ↓             ↓              ↓
  Symbol FST         Refs FST        BM25 doc       HNSW vectors  Call graph   Hierarchy     Clusters
  (structural)    (cross-file refs) (body tokens)   (semantic)   (callers FST / (v8 edges)   (v9 Leiden-
                                                                  callees CSR)                 CPM)
                                      ↓
                       Embedder trait → fastembed / MiniLM-L6 (default)

Per-project sidecars (in <index_dir>/):
  · index.vex             — primary index (format v9)
  · manifest.json         — metadata: embedder, sections, version, staleness tracking
  · index.bloom           — symbol-name bloom filter (`vex check` skips FST lookups for definitely-missing names)
  · index.trigram         — per-file trigram bloom so `vex grep` skips non-matching files (v1.24.1)
  · index.hnsw            — semantic vectors (HNSW graph)
  · index.bodytokens      — per-symbol terms for BM25 + semantic context (B1.2)
  · index.git_history     — historical symbol presence (Phase 14.8, FST + git-walk fallback)
  · index.rename_chains   — MinHash+LSH rename tracking across commits (Phase 14.10)
  · index.state           — incremental state: imported_by reverse map + writer-provenance sentinels (audit C1)

Shared cross-project (in user cache root, e.g. ~/Library/Caches/vex/blobs/):
  · {sha}.bin shards      — content-addressed parse cache, keyed by git blob SHA (Phase 14.7)

Search pipeline:
  search   → Symbol FST + BM25 + HNSW  → N-way RRF fusion → Hybrid tag on cross-channel hits
  usages   → Refs FST + posting lists → zero-copy, --strict adds type-aware filter (v1.14.1)
  history  → walks git tree, follows rename chains via Jaccard + greedy 1:1 (Phase 14.10)
  similar  → HNSW nearest neighbors (hash-keyed for content-stable IDs)
  show     → tree-sitter node boundaries → symbol body extraction
  pattern  → AST-aware structural matcher with skeleton index

• No SQLite — custom binary format v6, zero-copy mmap reads; readers accept v3+ for backwards compatibility
• Symbol FST — persistent inverted index, O(query_len) lookup
• Refs FST + ref_edges — symbol references as FST + cross-file edges resolved at write time (Pass-2 in store::writer); enables refactor-grade usages --strict
• Persistent call graph — CallEdge records + a name-keyed callers FST + a dense callees CSR index (v9+; FST on older indexes), built at index time, ~4ms lookup vs seconds of live tree-sitter scan
• BM25 channel — Okapi BM25 over body_tokens, auto-on when section present
• HNSW — approximate nearest neighbor via usearch, O(log N) semantic search; hash-keyed entries for content-stable IDs across re-indexing
• Pluggable embedder — Embedder trait + registry, identity recorded in manifest with mismatch detection at search
• History index — symbol presence per commit + MinHash-based rename chain tracking (closes LIMITATIONS §4c #2 for 1:1 renames)
• Parallel parsing — rayon with 500-file chunks; blob-SHA parse cache (shared across projects in the user cache root) skips re-parse of unchanged files across re-indexes
• Incremental updates — content hashing via xxh3; vex update re-parses only changed files (unchanged symbols + call edges reconstructed from existing index)
• Watch mode — notify crate with 500ms debouncing
• N-way RRF fusion — fuse_many merges structural + BM25 + semantic ranked lists, marks cross-channel hits as Hybrid
• Ranking eval harness — vex eval over a bundled golden set; CI regression gate on mean nDCG@10 + per-query-type floors + per-channel attribution (Phase 13.12 / 13.12.1)

License

MIT