smart-context-mcp

MCP server that reduces AI agent token usage by up to 90% through intelligent context compression (measured on this project).

What it is

An MCP (Model Context Protocol) server that provides specialized tools for reading, searching, and managing code context efficiently. Instead of loading full files or returning massive search results, it compresses information while preserving what matters for the task.

Real metrics from production use:

• ~7M tokens → ~800K tokens (approximately 89% reduction)
• 1,500+ operations tracked across development
• Compression ratios: 3x to 46x depending on tool
• Context overhead is tracked separately so reports can show gross and net savings

Workflow-level savings:

• Debugging: ~85-90% token reduction
• Code Review: ~85-90% token reduction
• Refactoring: ~85-90% token reduction
• Testing: ~85-90% token reduction
• Architecture: ~85-90% token reduction

Real adoption in non-trivial tasks:

• Approximately 70-75% of complex tasks use devctx tools
• Most used: smart_read (850+ uses), smart_search (280+ uses), smart_shell (220+ uses)
• Primary reasons for non-usage: task too simple, no index built, native tools preferred

See Workflow Metrics and Adoption Metrics for details.

Latest Release: 1.21.0

Minor release built around a semantic engine. MCP grows from 20 → 22 tools (smart_code + smart_output). SQLite schema bumps 8 → 9 (new outputs table; auto-migrates on first run). Zero new runtime dependencies — the TypeScript LanguageService comes from the typescript package you already have, and degrades to a fallback provider when it isn't there.

• smart_code (new tool). Semantic code navigation for JS/TS via the TypeScript LanguageService, resolving by symbol name (plus optional filePath) instead of forcing you to know a line and column. Actions: definition, references, implementations, diagnostics, impact, rename. Returns compact locations (file + 1-based start/end), never full file bodies. includeTests=false drops test paths; maxResults caps the payload.

• Impact analysis. smart_code(action='impact') splits direct hits (semantic: definitions, references, implementations) from transitive files (import graph, maxHops default 2) and related tests, plus coverage flags. risk.level is labelled basis: 'heuristic' and says so in its own note — it is a ranking aid, not semantic certainty.

• Scope-aware rename, dry run by default. smart_code(action='rename') uses findRenameLocations and dryRun: true is the default: the first call returns the planned per-file diff hunks (before/after per line) and writes nothing. Error conflicts (invalid or reserved newName, same name, unresolved target, path escape, missing file, blast radius over maxFiles) always block the write. name-collision is a heuristic warning that only blocks with strict: true, because the check doesn't verify scope overlap. After a real write, diagnosticsAfter reports remaining TypeScript errors in the touched files. smart_edit remains the tool for non-semantic textual replacements.

• smart_output (new tool). Persistent output/artifact store (shell, test, build, lint, diff) so a stack trace from twenty turns ago is recoverable without rerunning the command. Actions: save, search, excerpt, summary, list, stats, prune. Content is scrubbed for likely secrets before persistence and truncated head+tail above DEVCTX_OUTPUT_MAX_BYTES (256KB) so the failing tail always survives. Retention via DEVCTX_OUTPUT_RETENTION_DAYS (14) and DEVCTX_OUTPUT_MAX_PER_KIND (50). Identical output for the same kind+command is deduped into repeat_count. Automatic capture from smart_shell/smart_test is opt-in via DEVCTX_OUTPUT_STORE=true (then smart_shell returns an outputRef); explicit save/search always work. Degrades with degraded: true instead of throwing when SQLite is unavailable or locked.

• Opt-in semantic expansion in smart_context. include: ['semantic'] expands context through real definitions, references and implementations rather than text similarity, and every item gains a whyIncluded explanation. Off by default until the precision@5 benchmark justifies otherwise.

• Richer ADR/spec awareness and graph paths carried over from the same roadmap: ADR sections are indexed as kind='adr' / 'adr-section', and smart_context can return graph paths between two files or symbols.

• Shared tokenBudget across tools. smart_read, smart_read_batch, smart_context, smart_turn (start + end) and smart_resume now accept tokenBudget: number | { id?, maxTokens, shared? }. When shared:true (or id set), the budget is reused across calls inside the same task — so a multi-step agent flow can stay under a hard token ceiling without per-call bookkeeping. Responses include taskBudget, remainingBudget, and budgetDetails (scope, actions, degraded mode) when the budget actually changed the output.

• smart_search search modes. New mode: 'needle' | 'balanced' | 'semantic' (default balanced). needle = literal exact only (no regex / no term expansion) — kills noise on debug queries. balanced = exact + regex + term expansion. semantic = exact-first plus the local semantic block only when exact signal is weak. The previous semantic: true flag remains as a legacy alias for mode: 'semantic'. Default maxFiles tightened 15 → 5. New maxTokens caps the whole response and compacts intelligently (matches first, then diagnostics, then semantic block). Per-file ranking is now inspectable via matchedBy, boostSource, scoreBreakdown, whyRanked. Response also returns hasMore / totalFiles / nextSuggestedMaxFiles and actionable suggestions when the query is too broad or empty.

• smart_read persistent cache + budget-aware full degradation. New SQLite read_cache table keyed by (filePath, mode, selector, content_hash). Second read of an unchanged file is virtually free. Mode full is now an explicit last resort: if a tokenBudget/maxTokens is set, it degrades to lighter modes first (outline → signatures → truncated) and reports the real mode used in fullMode + budgetDetails. New clearReadCachePersistent + GC integration in runStorageMaintenance.

• smart_turn simple-task skip heuristic. When the prompt is short (≤ 40 chars after normalization), classified as a simple task, and no session/task is pinned, smart_turn(start) now returns skipSmartTurn: true with recommendedPath.mode='simple_task_skip' instead of paying the full orchestration cost. Saves continuity-resolution overhead on trivial prompts. minimal verbosity additionally compacts summary/refreshedContext to the fields agents actually consume.

• global_memory noise hints. Per-project, scrubbed noise telemetry persisted to ~/.devctx/global.db (noise_hints table). New actions noise_stats and noise_reset (full or via query). Lets smart_search learn which queries the agent already discovered to be noisy in a given repo and adjust ranking, without ever leaking content.

• KPI baseline infrastructure. New scripts evals/kpi-baseline.js + evals/kpi-utils.js aggregate harness.js and realworld-eval.js runs into a single JSON snapshot with top-5 precision, recall, reread task/call rate, and per-task-size buckets (short / long). Persists kpi-baseline-latest.json for regression detection across releases. New test suite tests/eval-kpis.test.js.

Highlights from 1.20.0 (still current)

Same tool count as 1.19.0, but several tools gained hard token-budget control, search-mode discipline and second-read cache reuse.

• Shared tokenBudget across tools. smart_read, smart_read_batch, smart_context, smart_turn and smart_resume accept tokenBudget: number | { id?, maxTokens, shared? }, reusable across calls inside one task so a multi-step flow stays under a hard ceiling.
• smart_search modes. mode: 'needle' | 'balanced' | 'semantic' (default balanced), maxTokens capping the whole response, and inspectable ranking via matchedBy / scoreBreakdown / whyRanked.
• smart_read persistent cache. SQLite read_cache keyed by (filePath, mode, selector, content_hash); a second read of an unchanged file is virtually free. Mode full degrades to lighter modes under a budget and reports the real mode used.
• smart_turn simple-task skip on short trivial prompts, and global_memory noise hints that let smart_search learn which queries proved noisy per repo.
• KPI baseline infrastructure (evals/kpi-baseline.js) snapshotting precision@5, recall, reread rate and latency for regression detection.

Highlights from 1.19.0

Five-step quality jump executed as sequential commits with full dogfooding. MCP grew from 18 → 20 tools, +68 tests, zero new dependencies.