# ooples/token-optimizer-mcp [Health: Active] **Category:** πŸ’° Finance & Fintech **Repository:** https://github.com/ooples/token-optimizer-mcp **GitHub Stars:** 516 **Views:** 1 **Installs:** 0 **Upvotes:** 0 **Directory Page:** https://allmcps.com/mcp/ooples-token-optimizer-mcp ## Description Intelligent token optimization achieving 95%+ reduction through caching, compression, and 80+ smart tools for API optimization, code analysis, and real-time monitoring. ## Claude Desktop Quick Installation Install path detected from listing signals. Uses `npx` (confidence: high): ```json "mcpServers": { "token-optimizer-mcp": { "command": "npx", "args": ["-y","token-optimizer-doctor"] } } ``` ## Documentation & README [![MCP Toplist](https://mcptoplist.com/badge/io.github.ooples%2Ftoken-optimizer-mcp.svg)](https://mcptoplist.com/server/io.github.ooples%2Ftoken-optimizer-mcp)

Token Optimizer MCP

Spend less context, keep the conclusions, and audit every claim across 16 coding clients.

npm version CI MIT license Node.js 22+

Measured on two independent harnesses 16 clients Direct savings measured before and after No telemetry MIT, commercial use allowed

Live Token Optimizer dashboard separating verified net MCP transport savings, excluded reports, per-agent attribution, and graph evidence

One local ledger for optimizer tools, live-graph substitutions, every agent, and the graph's own cost.

--- ## The 30-second version Your agent burns most of its context on work it already did: re-reading files that have not changed, dumping a whole file to see three lines, running unbounded searches, and re-deriving conclusions it reached last session and then forgot. Token Optimizer attacks that on four fronts. **1. It makes the expensive call impossible.** Install the plugin and a built-in `Read` of a 200 KB file is **denied**, with the refusal naming the cached, diffed replacement. Same for `Grep`, `Glob`, `Edit`, `Write`, and `cat` / `head` / `grep -r` through the shell. **Re-reading a file you already read this session returns only a diff** β€” usually the single biggest win, and one that size-based rules structurally cannot catch. There is no setting to turn on. **2. It remembers what your agent worked out.** A per-project knowledge graph accumulates findings, decisions and dead ends as a side effect of working, then feeds them back the moment the agent touches the relevant file. A finding costs ~150 tokens to carry. Re-deriving it costs 5k–50k. **3. It measures itself, in public, and tells you when it is losing.** A materialized before/actual-return measurement for MCP progressive disclosure, with later expansions debited from the same net. Modeled graph substitutions and the randomized control arm for downstream graph effects remain separate. Every number is measured, visibly collecting, excluded, or absent. **4. It attributes the traffic.** Returned context, optional cost equivalents, and net transport avoided are grouped by operation and MCP handshake identity. Codex, Claude Code, Gemini, and any other connected client get separate rows. Old records without identity remain explicitly unattributed instead of being assigned to whichever agent happens to be open now. No account, no telemetry, no hosted service. MIT, so it is usable at work. ### What the dashboard proves on a real machine The screenshots in this README come from the shipped server reading persisted local data, not a design mockup. In the capture above it reports: - **43,491 net verified MCP transport tokens avoided** in the current live proof: 54,037 gross reduction minus a deliberate 10,546-token expansion; - **486,074,740 historical/tool-reported tokens quarantined**, dominated by repository scan volume that never entered model context; - a live Codex / Claude Code / Gemini stdio smoke against AiDotNet, with each client attributed independently; - **2,648 graph nodes, 6,527 edges, and 58 findings** across 11 local projects; - **more than 1,000 hook runs with zero failures and zero timeouts** across six active CLI clients in the selected rolling 24-hour window; - **6,332 tokens of modeled graph-substitution potential**, excluded from the verified headline while the causal graph-reuse study remains `Collecting`. The dashboard now reads native CLI usage receipts and prices uncached input, cache reads, cache writes, and output with the exact captured provider, model, route, request-time tier, and versioned official source. Ambiguous model ids stay **Not priced** instead of receiving a blended guess. API/list-price equivalents are kept separate from provider-reported charges and are never labeled as a subscription invoice. See the [token accounting contract](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/TOKEN_ACCOUNTING.md). ## Quick start **Claude Code** β€” install the plugin, not the bare MCP server. The plugin is what enforces; adding the server alone just gives the model tools it can ignore. ```text /plugin marketplace add ooples/token-optimizer-mcp /plugin install token-optimizer@token-optimizer /reload-plugins ``` That is the entire installation. [All sixteen clients β†’](#installation) Then, whenever you want to know what to do next: ```text token_audit ``` One ranked queue: what is costing the most per session, with an optional monthly cost equivalent only after you configure your own effective rate. Each line names how to fix it. Not a dashboard, not six reports β€” a queue. --- ## The knowledge graph β€” the part nothing else has Every agent session ends the same way: the reasoning evaporates. The next session re-derives it, at full price, forever. This builds a **living per-project graph** β€” nodes for files, symbols, tasks and findings; edges for `derived_from`, `contains`, `supersedes`, `contradicts`, `related` β€” and it fills itself in from real work. No ingestion job, no embedding model, no rebuild step, no query to formulate. ``` you touch src/auth.ts β”‚ β”œβ”€ verify() compares exp against the LOCAL clock (finding, 0.9) β”œβ”€ per-host retry budgets; global was rejected β€” deadlock (decision) β”œβ”€ ! the skew fix was reverted once already (dead end) └─ [git] 47 changes in 90d, last three: "fix token expiry", "revert skew fix", "fix token expiry again" ``` None of that is in your repository. It exists only because an agent once burned tokens finding it out β€” and every other tool throws it away at the end of the session. **What a default install actually produces.** The structural graph β€” files, symbols, tasks, and the edges between them β€” is captured from ordinary tool traffic with no configuration at all. Findings are produced two ways. At session end, `derive` reads evidence already on disk (command outcomes and exit codes, red-to-green transitions, corrections, re-read churn) and writes findings from it: no model call, no credential, nothing sent anywhere. And the active model records durable conclusions itself through `wiki_write`. The **model-based semantic harvest** is the third path, and the only one that needs something you do not already have. It is **not opt-in** β€” `TOKEN_OPTIMIZER_HARVEST=0` turns it *off* β€” but its real gate is a credential: with none it reports `off:no-key`, which is the state on CI, corporate machines, and subscription-only logins. Point `TOKEN_OPTIMIZER_HARVEST_ENDPOINT` at a local model and it runs **free and private, with nothing leaving the machine**. `npx token-optimizer-doctor` states which of these is live. ### Why this is not RAG | Classic RAG | This | | -------------------------------------------------------------- | ----------------------------------------------------------------------------- | | Retrieves **evidence**; the model re-derives meaning each time | Retrieves **verdicts** β€” the reasoning already happened | | Index built by a **batch ingestion job** | Accretes from **real agent traffic** β€” coverage follows attention | | **Similarity** search | **Traversal** β€” this symbol _and its callers_ | | Model must **formulate a query** | Fires when the model **reaches for a file** | | Staleness **invisible**; serves rotted chunks confidently | Staleness **computed** from content hashes, served with the invalidating diff | | Returns only what is **in the documents** | Returns **dead ends**, which exist nowhere in your source tree | Traversal plus lexical search: deterministic, instant, explainable, and it works offline. ### The zero-turn refusal A plain deny costs a full turn: the model calls `Read`, is refused, re-plans, calls `smart_read`. But at refusal time we already hold the file _and_ the snapshot the graph stored β€” so the refusal **carries the answer inside it**. Nothing to re-plan, no second call. Turn cost drops from one to zero. And when the graph already holds the verdict a tool output would support, the output never enters context at all. Not compressed. Absent. --- ## The dashboard ```bash npm install npm run build npm run dashboard # http://localhost:3100 ```

Per-agent token accounting with historical rows left unattributed and live Codex, Claude Code, and Gemini rows measured separately

The overview answers the questions a token optimizer should answer first: 1. **How much MCP context did it avoid?** The headline is gross materialized payload reduction minus every later linked expansion. Graph estimates are intentionally separate. 2. **How much context still reached the agents?** Every successful current MCP result records its actual returned text, even when no valid before-state exists. That row is context-accounted but savings-unmeasured. 3. **Which agent and action spent it?** The client ledger and action table show operations, returned context, optional cost equivalent, and net tokens avoided. Lifecycle-only clients say `Not measured`; no zero is invented. 4. **Did remembering cost more than it saved?** Delivery and semantic-harvest tokens are charged to the graph. A causal benefit is added only after the treated/holdout evidence gate passes. ### Walkthrough: get useful data, not an empty dashboard 1. Install the MCP server and the native adapter for your CLI. The MCP handshake provides per-client accounting; native lifecycle hooks provide automatic routing, capture, health, and delivery where the client protocol permits it. 2. Use `smart_read`, `smart_grep`, `smart_glob`, `smart_edit`, or any other MCP operation normally. Every successful result records returned context; tools with a comparable materialized before-state also record a gross reduction; later `expand` calls debit that reduction. 3. Let the active model record durable conclusions with `wiki_write`. Before a new agent re-derives work, call `wiki_read` for the project or the files it is about to touch. Native clients can also deliver matching knowledge automatically. Use `wiki_query` to read the graph directly β€” one finding by key, a ranked BM25 search over claims, a node with its neighbours, or the graph's own audit β€” which is how a subagent that never sees the SessionStart briefing reaches what previous sessions established. 4. Open `http://localhost:3100`. Use **Overview** for combined accounting and **What it knows** for capture health, graph exploration, audits, and causal evidence. 5. To register existing local repositories without reading their source, run `npm run projects:discover -- /absolute/path/to/repos`. This makes coverage gaps explicit; it does not fabricate findings. For maintainers, this live smoke exercises the shipped stdio transport and creates separately attributed rows without seeding the analytics database: ```bash npm run dashboard:attribution-smoke -- /absolute/path/to/project /absolute/path/to/large-file npm run dashboard:verify-live -- http://localhost:3100 ```

Structured cross-client hook and MCP health cards from live local data

The health panel is deliberately operational rather than a raw text dump. Each client has activity, runtime failures/timeouts, policy outcomes, and observed surface coverage. Diagnostics keep no prompts, commands, paths, or tool output.

Direct graph savings, remembering cost, holdouts, and an honest collecting causal study

Modeled substitution potential and causal graph effects are different claims. The first is a full-file counterfactual and is never promoted to the verified MCP headline. The second asks whether delivered knowledge prevented later reads; it uses a control arm and remains `Collecting` until there are at least 20 treated file touches and 5 holdouts with valid downstream joins.

Interactive 3D knowledge graph spanning eleven local projects

Drag to orbit, scroll to zoom, click a node for provenance, or switch to the bounded one-hop focus view. The default **All known projects** scope pools local graphs through opaque project IDs; filesystem paths never reach the browser. **Audit tab.** Contradictions, stale findings, and low-confidence claims remain reviewable instead of silently becoming model truth. **Evidence console.** Client/model/task cohorts, matched effects with 95% intervals, live outcome joins, harm feedback, and capability tiers for all 16 clients. Release and superiority claims fail closed while evidence is missing. **One-click Markdown export.** The accumulated graph becomes documentation you can inspect, edit, and commit. Server-side by design: the browser asks for a neighbourhood, a search result or a page. A mature graph holds thousands of nodes, and shipping it wholesale would make every page load a multi-megabyte download for a view that shows twenty things. The default **All known projects** scope combines captured graphs through an opaque machine-local project registry; filesystem paths never reach the browser. Lifecycle hooks register repositories as they are used. To backfill existing local checkouts without reading their source files, run the bounded discovery command against one or more explicit roots: ```bash npm run projects:discover -- /absolute/path/to/repos /absolute/path/to/worktrees ``` Coverage distinguishes repositories with graph data from known repositories whose capture has not started. The balance cards are backed by persisted events: **Memory deliveries** counts graph context actually supplied to an agent, **Kept back for comparison** counts randomized control touches, and **Cost of remembering** combines delivered-context tokens with measured semantic-write payload cost. **Reading avoided** stays `Collecting` or `Not measured` until at least 20 treated file touches and 5 holdouts exist with a downstream join; the dashboard does not manufacture a savings estimate from missing data. See [the causal evidence protocol](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/EVIDENCE_PROTOCOL.md), the [cross-client capability contract](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/CLIENT_SUPPORT.md), and the [live evaluation suite](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/evals/README.md). ### Cross-client lifecycle diagnostics Every native hook writes the same bounded JSONL lifecycle record, including Claude Code's custom router and compaction paths. Records carry the client and plugin versions, event, hashed session/turn correlation, latency, outcome, input/output byte counts, and response key shape. They deliberately retain no prompt, command, tool output, file content, or raw working-directory path. The fields include OpenTelemetry log severity and resource semantics so the local files can be collected without inventing a second schema. ```bash npm run diagnostics # last 24 hours, summary-first JSON npm run diagnostics -- --hours 72 --output hook-summary.json npm run diagnostics -- --include-events --limit 100 --output hook-report.json ``` Raw event rows are opt-in and capped at 1,000. The default report contains aggregate health and at most twenty recent failures/timeouts, keeping routine troubleshooting output small enough for CLI and model context windows. The dashboard's **Capture health** panel shows runs, failures, timeouts and p50/p95 latency by client. Logs rotate at 5 MiB, retain at most 40 files for 14 days, and live under `.token-optimizer/logs` when a state directory is set (or `~/.token-optimizer/logs` otherwise). `TOKEN_OPTIMIZER_LOG_DIR`, `TOKEN_OPTIMIZER_LOG_MAX_BYTES`, `TOKEN_OPTIMIZER_LOG_MAX_FILES`, and `TOKEN_OPTIMIZER_LOG_RETENTION_DAYS` override those operational defaults. --- ## What it does that other optimizers do not ### Compaction is consolidation, not loss Everyone else checkpoints and restores what you _had_ β€” which spends the scarcest budget in the session replaying context you already paid for. Selection here is **derived**, not a category list: `cost-to-rederive Γ— irrecoverability Γ— reuse-probability`, with dead ends and decisions on a floor, because cheap-to-find is not the same as cheap-to-find-_again_. Restoration then adapts to the situation β€” mid-problem, cold resume, or in-flow β€” within a measured budget: > **Where you were:** does clock skew explain the 401s? > ruled out: token signing, clock drift on the client > untested: NTP skew on the server That is resuming a thought. A summary describes one. ### Progressive disclosure that knows what you asked A large tool result becomes a preview chosen by the **session's actual question**, after parsing the output's shape (test report, diff, stack trace, log, JSON) β€” not the first 40 lines because they are first. ``` [selected against: "which shard fails?"] --- failures --- FAILED DBNetTests.BceOnRelu -- expected 0.0 got NaN FAILED TftGradientFlow -- gradient did not reach the encoder ---- omitted: 1,760 lines of passing tests (expand 8bb6bd66) ---- ``` Every cut is **named**, because a model reasoning over a silent truncation cannot know it is missing anything. `expand` serves from a content-addressed store β€” it never re-runs your test suite β€” and expanding both teaches the next preview and promotes what you needed into the graph, so the second expansion never happens. ### Prompt-cache economics, measured from your own transcript Provider caches are billable and provider-specific; a cache hit is not a free input token. For example, Anthropic publishes separate cache-read and cache-write multipliers, while OpenAI and Gemini expose their own cached-input usage and pricing rules. Token Optimizer reads native cache fields when the client supplies them and keeps reads, writes, uncached input, and output separate. It never applies one provider's cache multiplier to another client. The attribution view then does the part a hit rate cannot: ``` ! CLAUDE.md:2 has an embedded timestamp, invalidating everything after it about 329,421 tokens re-written per session ``` Attributed to a line, priced by what sits _behind_ it. Keep-warm is decided by expected value from your observed gaps, per TTL tier β€” and when neither tier pays, it says so. ### Model routing decided by outcomes, not by task size Everyone guesses from task shape and never checks. This reads which model ran each episode and what happened β€” retries, errors, turns β€” and prices **both** mistakes: what an overpowered model wastes, and what an underpowered one costs in retries. A tier that needs a retry in more than half its episodes is excluded at any price, because four cheap turns that fail are not cheap. ### Waste detection that becomes a ratchet A report is read once and forgotten. Here a detection produces a **durable, measured, reversible fix** β€” a skip rule, a composite touch β€” plus a ~50-token session-start briefing so the waste never starts. Detectors are a shipped floor _plus_ patterns derived from your project's own history, each carrying what it has actually saved: ``` generated/schema.d.ts: read in 9/9 sessions, never the source of a finding 3,400 tokens/session; cost equivalent not priced; apply: waste_audit action="apply" ``` Anything that touches **your** files is proposed as a diff and never applied. ### One audit across every project `fleet_audit` ranks your whole machine by measured cost, and does something a per-project scanner cannot: a fix proven in one project is offered to the others containing the **same file contents**, carrying the evidence from where it was measured. Matching is by content hash, never by filename. It also runs the natural experiment nobody else can β€” enforcing clients versus directive ones β€” and reports it whichever way it falls, with the confound stated. --- ## Trust: we ship hooks that refuse your tool calls That is a bigger ask than a normal dependency makes, so: **Verify the release.** Published from CI with npm provenance β€” `npm audit signatures` ties the artifact to the workflow run and the commit, without trusting us. `CHECKSUMS.sha256` ships alongside for offline checking. **If nothing seems to be happening, the lifecycle bundle is probably not installed.** An MCP server process cannot modify the host that launched it, and npm 11 gates lifecycle scripts behind `allow-scripts`. Install the native plugin/hook bundle listed for your client below; adding only the MCP server gives the model tools but no pre-execution veto. For a legacy global Claude Code installation, recovery is one line: ```bash npx token-optimizer-install # wire Claude Code hooks npx token-optimizer-doctor # prove the Claude hooks work ``` **Check that it works β€” not that files exist.** ```bash npx token-optimizer-doctor # or npm run doctor, from a clone ``` It feeds a synthetic payload to the _real_ hook binary and asserts a large read is refused and a small one is not. A checklist would have passed on the exact bug this project once shipped, where the plugin was connected, visible in `/mcp`, and saving nothing. Every failure names its own fix. **Every refusal carries its own off switch.** Enforcement that hides its disable is coercive, and the person who needs it is mid-refusal, not reading a README: ``` auth.ts is 91 KB. Call smart_read instead. (Not what you wanted? TOKEN_OPTIMIZER_MODE=off disables enforcement.) ``` **Removal is exact.** The installer records every file it wrote, with hashes. Uninstall removes only what still matches; anything you edited since is left in place and named. Your own hooks are never touched, and we never rewrite your `settings.json` β€” we merge into it. ```bash npm run uninstall-hooks # show the plan; changes nothing npm run uninstall-hooks -- --apply # carry it out ``` --- ## It separates direct savings from causal graph evidence Every tool in this space reports "tokens saved" computed from its own assumptions. That number cannot be wrong, because nothing checks it. For optimizer results, this records the materialized before-state and the text actually returned to the client, then subtracts any linked expansion responses. Native graph-substitution counterfactuals stay labeled as modeled and outside the main headline. The graph's broader claimβ€”whether delivered knowledge prevented later re-readingβ€”is causal, so it runs a **randomized holdout**. Delivery is silently withheld on a slice of file touches, stratified by file, and the effect is the difference in downstream reads between arms. That effect is not added to the combined net until the experiment can support it, and the page will tell you plainly: > the graph is NOT yet paying for itself A tool that can only ever report good news is not reporting. The same discipline runs throughout: an unmeasurable saving renders as **unknown**, never as zero, and never as `$0.00` β€” because "cannot tell yet" printed as "saved nothing" is a silent false negative, and dollars get quoted to other people. ## It runs everywhere, and says which tier The tier is a protocol guarantee, not a preference: - **Lifecycle continuation:** Claude Code, Codex, GitHub Copilot CLI, Gemini CLI, Qwen Code, and Cursor. Native routing/capture/delivery plus one active-model completion reflection. - **Native observation:** Cline, OpenCode, Kilo, and Windsurf. Native routing/capture/delivery; the active model performs semantic writes. - **MCP + rules:** Roo Code, Zed, Amp, Continue, Crush, and Droid. MCP-visible activity and explicit graph tools, with no claim over hidden built-in calls. The exact surfaces still differ. For example, a protocol that can replace a large read before it reaches the model provides a stronger token guarantee than one that can only observe it. The generated registry, adapters, dashboard, and certification report all read the same capability source so those claims cannot drift independently. Every config shape is confirmed against that client's published documentation, with the source URL recorded in its README. ## Everything here is verified, and you can run it ```bash npm run verify:all ``` | Suite | Checks | What it proves | | ---------------- | ------ | ----------------------------------------------------------------------------------------------------------------------- | | `test` | 2,437 | Enforcement, staleness, injection, consolidation, disclosure, cache, routing, trust β€” driving the real hooks over stdin | | `verify:clients` | 253 | Every client config, lifecycle manifest, and enforcement surface matches its documented schema | | `verify:harvest` | 26 | Request shape, response parsing, and that **no secret from a tool result crosses the wire** | | `verify:ui` | 22 | Real headless Chromium: layout, label collisions, legibility | | `doctor` | 10 | The installed hooks actually refuse, and the server actually answers | These are not decoration. They found six client configs that would have failed **silently**, an installer that destroyed user hooks, a Windows path bug that made enforcement blind to half its own refusals, and an uninstaller that printed a plan and deleted nothing. ## Honest comparison | | Token Optimizer | Typical alternatives | | ---------------------------- | -------------------------------------------------------------------- | ---------------------------------------------------- | | **License** | MIT β€” commercial use fine | Often noncommercial-only; check before using at work | | Default behaviour | Refuses the wasteful call | Suggests a better tool | | Re-read of an unchanged file | Returns a diff | Returns the file again | | Savings figure | Direct before/return ledger; causal graph effect separately held out | Computed from the tool's own assumptions | | Cross-session memory | Findings, decisions, dead ends | Usually none | | Compaction | Consolidation, ranked by cost-to-rederive | Checkpoint and replay | | Cache economics | Measured from the transcript, attributed to a line | Rarely addressed | | Model routing | Measured from episode outcomes | Guessed from task size | | Cross-project | Fixes transfer by content hash | Per-project only | | Clients | 16 | 3–6 typical | | Telemetry | None | Varies | --- ## Installation Every client launches the same stdio server β€” `npx -y @ooples/token-optimizer-mcp@latest` β€” but they differ in what they let a hook _do_, and that difference is the whole product. A client with a pre-execution veto can have the wasteful call refused; one without can only be told. Both are listed honestly below. Ready-made configuration for all sixteen lives in [`integrations/`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations), generated from one source and validated by `npm run verify:clients`. Full matrix: [`docs/CLIENT_SUPPORT.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/CLIENT_SUPPORT.md). Every MCP connection also receives capability-aware mandatory routing instructions in its `initialize` response. That gives all clients a universal always-on policy, but only the ten clients with native pre-tool surfaces can hard-veto a wasteful built-in call; install their lifecycle bundle for actual enforcement. ### Enforcing tier β€” the wasteful call is refused These ten clients expose a pre-execution hook. Their packaged lifecycle bundle shares one capability-aware decision engine and defaults to `assist`: routing, retrieval, capture and harvest all on, no refusals. Set `TOKEN_OPTIMIZER_MODE=enforce` if you want expensive built-in calls vetoed, or `off` to disable the hooks entirely. | Client | Installable lifecycle surface | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | **Claude Code** | Native plugin: `/plugin marketplace add ooples/token-optimizer-mcp`, then `/plugin install token-optimizer@token-optimizer` | | **Codex** | Native plugin in [`integrations/codex/plugin`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/codex/plugin), or standalone hooks in [`integrations/codex`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/codex) | | **GitHub Copilot CLI** | Project hooks in [`integrations/copilot`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/copilot) | | **Gemini CLI** | Gemini extension at the repository root, backed by [`integrations/gemini`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/gemini) | | **Qwen Code** | Extension bundle in [`integrations/qwen`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/qwen) | | **Cursor** | Project hooks and always-applied rule in [`integrations/cursor`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/cursor) | | **Cline** | Project hooks and rule in [`integrations/cline`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/cline) | | **OpenCode** | In-process plugin and hooks in [`integrations/opencode`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/opencode) | | **Kilo** | In-process plugin and hooks in [`integrations/kilo`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/kilo) | | **Windsurf** | Project hooks and rule in [`integrations/windsurf`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/windsurf) | ### Rules tier β€” mandatory routing where the host has no veto API Roo Code, Zed, Amp, Continue, Crush, and Droid do not expose a packaged pre-execution bridge that can safely veto built-in calls. Their generated, always-on rules make optimized routing mandatory whenever the exact MCP schema is visible, and fail open to a bounded native operation when it is not. The integration directories contain both the MCP config and the rules file at the paths documented by each host. > **These paths are checked, not assumed.** Verifying them against each client's > published docs found six configs that would have installed cleanly and never > loaded β€” including Kilo, whose schema shares nothing with the `mcpServers` > convention the other clients use. A convention is not a schema. ### Codex #### 1. Add the MCP server or native plugin For the best experience, install the Codex plugin. It bundles the MCP server, the token-optimization skill, session guidance, and a large-read hook: ```bash codex plugin marketplace add ooples/token-optimizer-mcp codex plugin add token-optimizer@token-optimizer ``` Review and trust the bundled hooks with `/hooks`, then start a new conversation. If you prefer an MCP-only installation, use: ```bash codex mcp add token-optimizer -- npx -y @ooples/token-optimizer-mcp@latest ``` On Windows, if PowerShell blocks the `codex.ps1` shim, use the command launcher directly: ```powershell codex.cmd mcp add token-optimizer -- npx -y @ooples/token-optimizer-mcp@latest ``` This writes the server to `~/.codex/config.toml`. Codex CLI, the Codex IDE extension, and the Codex app on the same host share that configuration. #### 2. Verify the installation ```bash codex mcp get token-optimizer codex mcp list ``` ![Codex MCP list showing token-optimizer installed and enabled](https://raw.githubusercontent.com/ooples/token-optimizer-mcp/HEAD/docs/assets/codex-mcp-installed.png) Start a new Codex conversation after installation so the new tools are discovered. In an interactive CLI session, `/mcp` shows the tools available to the conversation. #### 3. Add optimization guidance and hooks The plugin supplies both automatically. For an MCP-only installation, add the guidance from [`integrations/AGENTS.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/AGENTS.md) to a project or global `AGENTS.md`. A ready-made standalone hook is also available under [`integrations/codex/hooks`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/codex/hooks); merge its `hooks.json` into `~/.codex/hooks.json`, copy the script to `~/.codex/hooks/`, and review it once with `/hooks`. The Codex hook injects guidance at `SessionStart`. Under `TOKEN_OPTIMIZER_MODE=enforce` it blocks expensive native operations when the bundled MCP has an exact replacement, including a single unambiguous code-mode shell call such as `cat` or `Get-Content`; multi-operation orchestration remains advisory so unrelated work is not discarded, and a second attempt at the same target passes through if the MCP is unavailable. The default is `assist`, which keeps retrieval and capture but never vetoes; `TOKEN_OPTIMIZER_MODE=advise` adds the routing advisory without vetoes, and `off` disables the hooks. The `AGENTS.md`/skill guidance remains important. If you prefer a smaller instruction block: ```markdown ## Token optimization Use the token-optimizer MCP for large or repeated reads: - `smart_read` for files over roughly 400 lines and for files already read once. - `smart_glob`/`smart_grep` for large search results. - `optimize_text` to store bulky text outside the model context. - `get_optimization_report` when the user asks for token or compression stats. Use normal tools for small, one-off operations. ``` See the current [Codex hooks documentation](https://learn.chatgpt.com/docs/hooks.md) for hook trust, matching, and tool-coverage details. #### Equivalent manual Codex configuration If you prefer to edit `~/.codex/config.toml` yourself: ```toml [mcp_servers.token-optimizer] command = "npx" args = ["-y", "@ooples/token-optimizer-mcp@latest"] # Optional: keep the cache in a custom location. # env = { TOKEN_OPTIMIZER_CACHE_DIR = "/absolute/path/to/cache" } ``` ### Claude Code **Install the plugin, not the bare MCP server.** The plugin is the only path that optimizes by default; adding the MCP server alone gives the model a set of tools it is free to never call. ```text /plugin marketplace add ooples/token-optimizer-mcp /plugin install token-optimizer@token-optimizer /reload-plugins ``` That is the whole installation. There is nothing to configure and no flag to turn on. #### What you get immediately From the first message of the next session, expensive built-in calls are **refused and redirected** to the optimized equivalent: | You (or the model) do this | What happens | | ---------------------------------------------- | ---------------------------------------------------- | | `Read` a file over ~25 KB | Denied β†’ `smart_read` (cached) | | `Read` **any** file already read this session | Denied β†’ `smart_read` (returns only the diff) | | `Grep` file contents / `Glob` for files | Denied β†’ `smart_grep` / `smart_glob` | | `Edit` a file over ~25 KB | Denied β†’ `smart_edit` (returns a diff, not the file) | | `cat`/`head`/`tail`/`Get-Content` a large file | Denied β†’ `smart_read` | | `grep -r` / `rg` across the tree | Denied β†’ `smart_grep` | | Context fills and compaction starts | `optimize_session` runs first | The re-read case is usually the largest single win and the one most often missed: a 5 KB config read fifteen times across a session costs far more than one 200 KB file read once. Size-based rules never catch it. #### It cannot get you stuck Three properties, all tested: - **Fail-open.** Any error in the optimizer β€” bad payload, unreadable file, unexpected exception β€” allows the original call through, exactly as if the plugin were not installed. - **Loop-breaking.** A given target is refused **once**. Come back to it and it is allowed. So if the MCP server is missing or misconfigured, the cost is one wasted turn per file, self-healing, with no intervention. - **Cheap calls are left alone.** Small files, paged reads, `git log | head`, and binary paths are never touched. #### Turning it down One variable, no reinstall: ```bash TOKEN_OPTIMIZER_MODE=advise # nudge instead of refuse (the pre-5.2 behaviour) TOKEN_OPTIMIZER_MODE=off # disable the hooks entirely TOKEN_OPTIMIZER_LARGE_READ_BYTES=51200 # raise the "large file" threshold ``` #### MCP server only (not recommended) If you want the tools without the enforcement: ```bash claude mcp add --transport stdio --scope user token-optimizer -- \ npx -y @ooples/token-optimizer-mcp@latest ``` Verify with `claude mcp get token-optimizer`, or `/mcp` inside Claude Code. Then add the recommendations from [`integrations/AGENTS.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/AGENTS.md) to your `CLAUDE.md` β€” but be aware that guidance in a context file is advisory, and models routinely read past it. The standalone global installer can also configure the Claude Code hooks and supported desktop clients: ```bash npm install -g @ooples/token-optimizer-mcp@latest ``` On Windows, a restrictive PowerShell policy may need this user-scoped adjustment first: ```powershell Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser npm install -g @ooples/token-optimizer-mcp@latest ``` Interactive global installs run the hook installer; CI and local dependency installs skip it. If automatic setup is skipped, use `install-hooks.ps1` on Windows or `install-hooks.sh` on macOS/Linux. See the [Claude Code MCP guide](https://code.claude.com/docs/en/mcp) and this project's [hook installation guide](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/HOOKS-INSTALLATION.md). ### GitHub Copilot CLI #### 1. Add the MCP server On current Copilot CLI releases: ```bash copilot mcp add token-optimizer -- npx -y @ooples/token-optimizer-mcp@latest ``` If your Copilot CLI does not expose `copilot mcp` yet, save this as `~/.copilot/mcp-config.json`: ```json { "mcpServers": { "token-optimizer": { "type": "local", "command": "npx", "args": ["-y", "@ooples/token-optimizer-mcp@latest"], "tools": ["*"] } } } ``` A ready-made copy is available at [`integrations/copilot/mcp-config.json`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/copilot/mcp-config.json). #### 2. Verify the installation ```bash copilot mcp get token-optimizer copilot mcp list ``` Inside an interactive Copilot session, `/mcp show token-optimizer` displays the connection status and available tools. #### 3. Add optimization guidance and hooks Keep [`integrations/AGENTS.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/AGENTS.md) as the repository's `AGENTS.md`, or adapt the same guidance into `.github/copilot-instructions.md`. For native lifecycle integration, copy the ready-made repository hooks into your project: ```bash mkdir -p .github/hooks cp integrations/copilot/.github/hooks/token-optimizer* .github/hooks/ ``` ```powershell New-Item -ItemType Directory -Force .github/hooks | Out-Null Copy-Item integrations/copilot/.github/hooks/token-optimizer* .github/hooks/ ``` The hooks inject optimization guidance at `sessionStart`. Under `TOKEN_OPTIMIZER_MODE=enforce` they deny a large built-in `view` so Copilot retries with `smart_read`; the default `assist` leaves the call alone. Partial reads and files below 25 KB always pass through unchanged, and `TOKEN_OPTIMIZER_MODE=advise` gives recommendations without vetoes. Repository hooks work without overwriting user-level files; global hooks can instead be placed in `~/.copilot/hooks/` with their script paths adjusted for that directory. Restart Copilot CLI after changing hook files. See GitHub's official [MCP setup guide](https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-mcp-servers) and [hooks reference](https://docs.github.com/en/copilot/reference/hooks-reference). ### Gemini CLI #### 1. Add the MCP server Add Token Optimizer directly at user scope: ```bash gemini mcp add --scope user token-optimizer npx -y @ooples/token-optimizer-mcp@latest ``` Alternatively, install this repository as a Gemini extension so the MCP configuration and `GEMINI.md` guidance are packaged together: ```bash gemini extensions install https://github.com/ooples/token-optimizer-mcp --auto-update ``` #### 2. Verify the installation ```bash gemini mcp list gemini extensions list ``` Run `/mcp` inside Gemini CLI to inspect the connection. Restart Gemini CLI after installing or updating the extension. #### 3. Add optimization guidance and hooks Direct MCP users should copy [`GEMINI.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/GEMINI.md) into the project or merge its rules into an existing `GEMINI.md`. Extension users receive that context file plus native hooks automatically. The extension's `SessionStart` hook injects optimization guidance. Its `AfterTool` hook notices full-file `read_file` results over 25 KB and suggests `smart_read`. To make Gemini automatically replace those large results with a token-optimizer tail call, configure the extension setting **Automatic large-read routing** as `true`: ```bash gemini extensions config token-optimizer ``` Automatic routing uses Gemini's native `tailToolCallRequest`: the `smart_read` result replaces the built-in read result before it reaches the model. Partial reads remain unchanged. Restart Gemini CLI after installing, updating, or reconfiguring the extension. See the official [Gemini MCP guide](https://geminicli.com/docs/tools/mcp-server/), [extension guide](https://geminicli.com/docs/extensions/reference/), and [hooks reference](https://geminicli.com/docs/hooks/reference/). ### OpenCode #### 1. Add the MCP server Create or update `opencode.json` in your project: ```json { "$schema": "https://opencode.ai/config.json", "mcp": { "token-optimizer": { "type": "local", "command": ["npx", "-y", "@ooples/token-optimizer-mcp@latest"], "enabled": true } }, "instructions": ["./AGENTS.md"] } ``` For a global installation, merge the same `mcp` entry into `~/.config/opencode/opencode.json`. #### 2. Verify the installation ```bash opencode mcp list ``` The output shows configured servers and their connection status. #### 3. Add optimization guidance and the local plugin Copy [`integrations/AGENTS.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/AGENTS.md) to the project as `AGENTS.md`; the `instructions` entry above loads it. Then copy the ready-made local plugin: ```bash mkdir -p .opencode/plugins cp integrations/opencode/.opencode/plugins/token-optimizer.js .opencode/plugins/ ``` ```powershell New-Item -ItemType Directory -Force .opencode/plugins | Out-Null Copy-Item integrations/opencode/.opencode/plugins/token-optimizer.js .opencode/plugins/ ``` The plugin preserves Token Optimizer usage state in OpenCode's compaction prompt. Under `TOKEN_OPTIMIZER_MODE=enforce` its `tool.execute.before` hook rejects full-file reads over 25 KB and steers the agent to `smart_read`; the default `assist` lets them through. Small and partial reads always pass normally, and `TOKEN_OPTIMIZER_MODE=advise` gives non-blocking guidance. Restart OpenCode after adding the plugin. See the official [OpenCode MCP guide](https://opencode.ai/docs/mcp-servers/) and [plugin hook guide](https://opencode.ai/docs/plugins/). ### Generic MCP configuration Any stdio-capable MCP client can launch Token Optimizer with: ```json { "mcpServers": { "token-optimizer": { "command": "npx", "args": ["-y", "@ooples/token-optimizer-mcp@latest"] } } } ``` Additional ready-made integration files are available for [Claude Desktop](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/examples/claude_desktop_config.json), [Codex](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/codex/config.toml), [Gemini CLI](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/gemini/), [OpenCode](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/opencode/), and [GitHub Copilot](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/copilot/mcp-config.json). ## Use it You normally use Token Optimizer by asking your agent in plain language: ```text Use token-optimizer smart_read for the large server file, then use it again after the edit so only the diff comes back. ``` ```text Cache this API response with optimize_text under the key customer-schema, then retrieve it only if we need the full payload again. ``` ```text Show my token savings with get_optimization_report. ``` For clients that expose direct MCP tool calls, the core inputs are small JSON objects: ```json { "tool": "smart_read", "arguments": { "path": "/absolute/path/to/large-file.ts" } } ``` ```json { "tool": "optimize_text", "arguments": { "text": "A large response, log, document, or generated artifact...", "key": "stable-reference-key", "quality": 11 } } ``` ```json { "tool": "get_optimization_report", "arguments": { "topN": 10 } } ``` ## Understand the compression stats `optimize_text` returns measurements with every call. This example uses a deliberately repetitive payload to make every field easy to see; it is not a benchmark: ```json { "success": true, "key": "customer-schema", "originalTokens": 4180, "compressedTokens": 72, "tokensSaved": 4108, "percentSaved": 99.55, "cached": true, "compressionUsed": true } ``` `get_optimization_report` applies the same versioned measurement contract as the dashboard and aggregates qualifying operations into: - gross preview reduction, linked expansion debits, and net transport avoided; - observed returned context and the net reduction percentage; - verified operations tracked, with legacy and tool-reported claims excluded; - breakdowns by action/tool, hook phase, and MCP server; - optional date-range and session filters. Two tools sound similar but serve different purposes: | Tool | Use it for | Context-window effect | | --------------- | ----------------------------------------------------------- | --------------------------------------------------------- | | `optimize_text` | Store bulky text under a key and return a compact reference | Reduces text kept in the active context | | `compress_text` | Produce Brotli/base64 data for storage or transport | May use **more** model tokens if pasted back into context | If your goal is a smaller prompt, prefer `optimize_text`. Use `compress_text` only when you specifically need byte compression. ## What is included | Capability | Representative tools | What gets smaller or faster | | ----------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------- | | Context and compression | `optimize_text`, `get_cached`, `count_tokens`, `analyze_optimization`, `context_delta` | Large payloads and repeated context | | File and Git operations | `smart_read`, `smart_write`, `smart_edit`, `smart_grep`, `smart_glob`, `smart_diff`, `smart_status` | File contents, search results, and diffs | | Caching | `smart_cache`, `cache_warmup`, `cache_invalidation`, `cache_compression`, `predictive_cache` | Repeated computation and retrieval | | APIs and databases | `smart_api_fetch`, `smart_sql`, `smart_graphql`, `smart_rest`, `smart_schema` | Responses, schemas, and query analysis | | Build and system tasks | `smart_build`, `smart_test`, `smart_lint`, `smart_logs`, `smart_processes` | Build logs and diagnostic output | | Intelligence | `smart-summarization`, `pattern-recognition`, `natural-language-query`, `recommendation-engine` | Analysis and summaries | | Analytics | `get_optimization_report`, `get_action_analytics`, `get_hook_analytics`, `export_analytics` | Token-savings visibility | See [`docs/TOOLS.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/TOOLS.md) for detailed tool inputs and examples. ## Requirements and data - Node.js 22 or newer - npm 9 or newer - An MCP client with stdio transport support Default local data locations include: - cache: `~/.token-optimizer-cache/` - analytics: `~/.token-optimizer-mcp/analytics.db` - sessions and configuration: `~/.token-optimizer/` Set `TOKEN_OPTIMIZER_CACHE_DIR` to override the cache location. ## Verify it, check it, remove it This installs hooks that **refuse your tool calls**. That is a bigger ask than a normal dependency makes, so here is everything needed to check it and undo it. **Verify the release is genuine.** The package is published from CI with npm provenance, which signs an attestation binding the artifact to the workflow run and the commit that built it: ```bash npm audit signatures ``` That verifies without trusting us. A `CHECKSUMS.sha256` is attached to each GitHub release for offline checking (`sha256sum -c CHECKSUMS.sha256`) β€” useful for mirrors, but weaker: it shares a trust root with the thing it hashes. **Check it actually works.** Not that the files are in place β€” that it _works_: ```bash npm run doctor ``` This feeds a synthetic payload to the real hook binary and asserts a large read is refused and a small one is not, that session-start emits the policy, that the graph directory is writable, and that the MCP server starts and lists its tools. Every failure names its own fix. There is also an `install_doctor` MCP tool. **Turn enforcement off, instantly.** Every refusal says this, so you never have to come back here to find it: ```bash TOKEN_OPTIMIZER_MODE=off # no enforcement, no hooks TOKEN_OPTIMIZER_MODE=advise # suggestions only, nothing is ever denied ``` **Remove it.** The installer records every file it wrote, with hashes, so removal is exact rather than best-effort: ```bash npm run uninstall-hooks # show the plan; changes nothing npm run uninstall-hooks -- --apply # carry it out ``` It removes only files that still match what we wrote. Anything you have edited since is **left in place and named**, because removing it would destroy your work and removing it silently would be worse. Hooks you added yourself are not in the manifest and are never touched. Config entries we added are listed for you to remove β€” we do not rewrite your `settings.json`. ## Technical reference The detailed operational material below is intentionally retained for users who want to understand the complete tool surface, hooks pipeline, performance controls, analytics, and troubleshooting behavior. The proposed category-level successor architecture, its 18 required workstreams, and the evidence gates for claiming a universal cognitive runtime are documented in [`docs/UNIVERSAL_COGNITIVE_RUNTIME_PROGRAM.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/UNIVERSAL_COGNITIVE_RUNTIME_PROGRAM.md). ### Complete Tool Reference (74 Total) #### Core Caching & Optimization (8 tools)
Click to expand - **optimize_text** - Compress and cache text (primary tool for token reduction) - **get_cached** - Retrieve previously cached text - **compress_text** - Compress text using Brotli - **decompress_text** - Decompress Brotli-compressed text - **count_tokens** - Count tokens using tiktoken (GPT-4 tokenizer) - **analyze_optimization** - Analyze text and get optimization recommendations - **get_cache_stats** - View cache hit rates and compression ratios - **clear_cache** - Clear all cached data **Usage Example**: ```typescript // Cache large content to remove it from context window optimize_text({ text: 'Large API response or file content...', key: 'api-response-key', quality: 11, }); // Result: 60-90% token reduction ```
#### Smart File Operations (10 tools)
Click to expand Optimized replacements for standard file tools with intelligent caching and diff-based updates: - **smart_read** - Read files with 80% token reduction through caching and diffs - **smart_write** - Write files with verification and change tracking - **smart_edit** - Line-based file editing with diff-only output (90% reduction) - **smart_grep** - Search file contents with match-only output (80% reduction) - **smart_glob** - File pattern matching with path-only results (75% reduction) - **smart_diff** - Git diffs with diff-only output (85% reduction) - **smart_branch** - Git branch listing with structured JSON (60% reduction) - **smart_log** - Git commit history with smart filtering (75% reduction) - **smart_merge** - Git merge management with conflict analysis (80% reduction) - **smart_status** - Git status with status-only output (70% reduction) **Usage Example**: ```typescript // Read a file with automatic caching smart_read({ path: '/path/to/file.ts' }); // First read: full content // Subsequent reads: only diff (80% reduction) ```
#### API & Database Operations (10 tools)
Click to expand Intelligent caching and optimization for external data sources: - **smart_api_fetch** - HTTP requests with caching and retry logic (83% reduction on cache hits) - **smart-cache-api** - API response caching with TTL/ETag/event-based strategies - **smart_database** - Database queries with connection pooling and caching (83% reduction) - **smart_sql** - SQL query analysis with optimization suggestions (83% reduction) - **smart_schema** - Database schema analysis with intelligent caching - **smart_graphql** - GraphQL query optimization with complexity analysis (83% reduction) - **smart_rest** - REST API analysis with endpoint discovery (83% reduction) - **smart_orm** - ORM query optimization with N+1 detection (83% reduction) - **smart_migration** - Database migration tracking (83% reduction) - **smart_websocket** - WebSocket connection management with message tracking **Usage Example**: ```typescript // Fetch API with automatic caching smart_api_fetch({ method: 'GET', url: 'https://api.example.com/data', ttl: 300, }); // Cached responses: 95% token reduction ```
#### Build & Test Operations (10 tools)
Click to expand Development workflow optimization with intelligent caching: - **smart_build** - TypeScript builds with diff-based change detection - **smart_test** - Test execution with incremental test selection - **smart_lint** - ESLint with incremental analysis and auto-fix - **smart_typecheck** - TypeScript type checking with caching - **smart_install** - Package installation with dependency analysis - **smart_docker** - Docker operations with layer analysis - **smart_logs** - Log aggregation with pattern filtering - **smart_network** - Network diagnostics with anomaly detection - **smart_processes** - Process monitoring with resource tracking - **smart_system_metrics** - System resource monitoring with performance recommendations **Usage Example**: ```typescript // Run tests with caching smart_test({ onlyChanged: true, // Only test changed files coverage: true, }); ```
#### Advanced Caching (10 tools)
Click to expand Enterprise-grade caching strategies with 87-92% token reduction: - **smart_cache** - Multi-tier cache (L1/L2/L3) with 6 eviction strategies (90% reduction) - **cache_warmup** - Intelligent cache pre-warming with schedule support (87% reduction) - **cache_analytics** - Real-time dashboards and trend analysis (88% reduction) - **cache-benchmark** - Performance testing and strategy comparison (89% reduction) - **cache_compression** - 6 compression algorithms with adaptive selection (89% reduction) - **cache_invalidation** - Dependency tracking and pattern-based invalidation (88% reduction) - **cache_optimizer** - ML-based recommendations and bottleneck detection (89% reduction) - **cache_partition** - Sharding and consistent hashing (87% reduction) - **cache_replication** - Distributed replication with conflict resolution (88% reduction) - **predictive_cache** - ML-based predictive caching with ARIMA/LSTM (91% reduction) **Usage Example**: ```typescript // Configure multi-tier cache smart_cache({ operation: 'configure', evictionStrategy: 'LRU', l1MaxSize: 1000, l2MaxSize: 10000, }); ```
#### Monitoring & Dashboards (7 tools)
Click to expand Comprehensive monitoring with 88-92% token reduction through intelligent caching: - **alert_manager** - Multi-channel alerting (email, Slack, webhook) with routing (89% reduction) - **metric_collector** - Time-series metrics with multi-source support (88% reduction) - **monitoring_integration** - External platform integration (Prometheus, Grafana, Datadog) (87% reduction) - **custom_widget** - Dashboard widgets with template caching (88% reduction) - **data_visualizer** - Interactive visualizations with SVG optimization (92% reduction) - **health_monitor** - System health checks with state compression (91% reduction) - **log_dashboard** - Log analysis with pattern detection (90% reduction) **Usage Example**: ```typescript // Create an alert alert_manager({ operation: 'create-alert', alertName: 'high-cpu-usage', channels: ['slack', 'email'], threshold: { type: 'above', value: 80 }, }); ```
#### System Operations (6 tools)
Click to expand System-level operations with smart caching: - **smart_cron** - Scheduled task management (cron/Windows Task Scheduler) (85% reduction) - **smart_user** - User and permission management across platforms (86% reduction) - **smart_ast_grep** - Structural code search with AST indexing (83% reduction) - **get_session_stats** - Session-level token usage statistics - **analyze_project_tokens** - Project-wide token analysis and cost estimation - **optimize_session** - Compress large file operations from current session **Usage Example**: ```typescript // View session token usage get_session_stats({}); // Result: Detailed breakdown of token usage by tool ```
#### Intelligence & Summarization (6 tools)
Click to expand - **intelligent-assistant** - Context-aware assistance with compact recommendations - **natural-language-query** - Natural-language querying over structured data - **pattern-recognition** - Pattern discovery with summarized findings - **predictive-analytics** - Predictive analysis with concise results - **recommendation-engine** - Ranked, context-aware recommendations - **smart-summarization** - Token-aware summarization for large content
#### Token Analytics (5 tools)
Click to expand - **get_optimization_report** - Complete savings report with totals and breakdowns - **get_action_analytics** - Savings aggregated by action or tool - **get_hook_analytics** - Savings aggregated by hook phase - **get_mcp_server_analytics** - Savings aggregated by MCP server - **export_analytics** - Export recorded analytics for external analysis
#### Context State & Storage (2 tools)
Click to expand - **optimization_storage** - Persist and retrieve optimized content - **context_delta** - Track compact context changes between states
### Architecture and Global Hooks #### Native client lifecycle coverage The MCP server is identical in every client, but lifecycle APIs are not. The repository ships client-native adapters instead of copying Claude event names into tools that would silently ignore them. | Client | Native integration events | Refusal under `MODE=enforce` | Advisory escape hatch | | ------------------ | ------------------------------------------- | ------------------------------------------------------ | ----------------------------- | | Codex | `SessionStart`, `PreToolUse` | Deny replaceable large reads and single shell dumps | `TOKEN_OPTIMIZER_MODE=advise` | | Claude Code | `PreToolUse` plus optional global pipeline | Deny replaceable large reads and noisy searches | `TOKEN_OPTIMIZER_MODE=advise` | | GitHub Copilot CLI | `sessionStart`, `preToolUse`, `postToolUse` | Deny large `view` calls and steer to `smart_read` | `TOKEN_OPTIMIZER_MODE=advise` | | Gemini CLI | `SessionStart`, `BeforeTool`, `AfterTool` | Deny replaceable large reads before they enter context | `TOKEN_OPTIMIZER_MODE=advise` | | OpenCode | `tool.execute.before`, compaction hook | Reject large full-file reads and steer to `smart_read` | `TOKEN_OPTIMIZER_MODE=advise` | The default is `assist`: routing, retrieval, capture and harvest are on and nothing is ever refused. That is the posture two independent harnesses measured as our best -- on THOL, assist scored 0.971 in 14.4 turns against control's 0.969 in 16.2, while enforce scored 0.960 in 20.3 turns and cost 1.471x control per task (median across 17 tasks; cheaper on only 2 of them). Set `TOKEN_OPTIMIZER_MODE=enforce` for the refusals described above, which use a 25,600-byte threshold; override it with `TOKEN_OPTIMIZER_LARGE_READ_BYTES`, use `TOKEN_OPTIMIZER_MODE=advise` for the routing advisory without vetoes, or `TOKEN_OPTIMIZER_MODE=off` to disable the lifecycle integration. Partial reads always pass through because they may already be more efficient than a full cached read, and under `enforce` one repeated attempt is allowed so a failed MCP server cannot permanently block work. #### Analytics workflow and storage
Click to expand Granular token usage analytics for pinpointing optimization opportunities: - **get_hook_analytics** - Token usage breakdown by hook phase (PreToolUse, PostToolUse, etc.) - **get_action_analytics** - Token usage breakdown by tool/action (Read, Write, Grep, etc.) - **get_mcp_server_analytics** - Token usage breakdown by MCP server (token-optimizer, filesystem, etc.) - **export_analytics** - Export analytics data in JSON or CSV format with filtering **Usage Example**: ```typescript // Get per-hook analytics get_hook_analytics({ startDate: '2025-01-01T00:00:00Z', endDate: '2025-12-31T23:59:59Z', }); // Result: Shows which hooks consume the most tokens // Get per-action analytics get_action_analytics({}); // Result: Shows which tools use the most tokens // Export analytics as CSV export_analytics({ format: 'csv', hookPhase: 'PreToolUse', }); // Result: CSV export filtered by PreToolUse hook ``` **Key Features**: - Per-hook phase tracking (PreToolUse, PostToolUse, SessionStart, etc.) - Per-action tracking (Read, Write, count_tokens, etc.) - Per-MCP-server tracking (token-optimizer, filesystem, GitHub, etc.) - Date range filtering - JSON and CSV export - Persistent storage with SQLite - Zero performance impact (async batched writes)
#### Global Hooks System (7-Phase Optimization) This complete seven-phase pipeline applies to the optional Claude Code global-hook installation. Codex, Copilot, Gemini, and OpenCode use the smaller native adapters above because their event names, payloads, and result-replacement capabilities differ. When global hooks are installed, token-optimizer-mcp runs automatically on **every tool call**: ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 1: PreToolUse - Tool Replacement β”‚ β”‚ β”œβ”€ Read β†’ smart_read (80% token reduction) β”‚ β”‚ β”œβ”€ Grep β†’ smart_grep (80% token reduction) β”‚ β”‚ └─ Glob β†’ smart_glob (75% token reduction) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 2: Input Validation - Cache Lookups β”‚ β”‚ └─ get_cached checks if operation was already done β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 3: PostToolUse - Output Optimization β”‚ β”‚ β”œβ”€ optimize_text for large outputs β”‚ β”‚ └─ compress_text for repeated content β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 4: Session Tracking β”‚ β”‚ └─ Log all operations to operations-{sessionId}.csv β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 5: UserPromptSubmit - Prompt Optimization β”‚ β”‚ └─ Optimize user prompts before sending to API β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 6: PreCompact - Pre-Compaction Optimization β”‚ β”‚ └─ Optimize before Claude Code compacts the conversation β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Phase 7: Metrics & Reporting β”‚ β”‚ └─ Track token reduction metrics and generate reports β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Performance evidence Reduction targets in tool descriptions are design goals, not production measurements. They never enter the verified ledger. Use the dashboard or `get_optimization_report` for request-level measurements, and `npm run dashboard:audit-savings` to inspect both qualifying and excluded rows. No universal dollar value is claimed: configure an effective rate only when it reflects your own provider, model, cache, route, plan, tier, and credits. ### Usage Examples #### Basic Caching ```typescript // Cache large content to remove from context window const result = await optimize_text({ text: 'Large API response or file content...', key: 'cache-key', quality: 11, }); // Result: Original tokens removed, only cache key remains (~50 tokens) // Retrieve later const cached = await get_cached({ key: 'cache-key' }); // Result: Full original content restored ``` #### Smart File Reading ```typescript // First read: full content await smart_read({ path: '/src/app.ts' }); // Subsequent reads: only changes (80% reduction) await smart_read({ path: '/src/app.ts' }); ``` #### API Caching ```typescript // First request: fetch and cache await smart_api_fetch({ method: 'GET', url: 'https://api.example.com/data', ttl: 300, }); // Subsequent requests: cached (95% reduction) await smart_api_fetch({ method: 'GET', url: 'https://api.example.com/data', }); ``` #### Session Analysis ```typescript // View token usage for current session await get_session_stats({}); // Result: Breakdown by tool, operation, and savings // Analyze entire project await analyze_project_tokens({ projectPath: '/path/to/project', }); // Result: Cost estimation and optimization opportunities ``` ### Technology Stack - **Runtime**: Node.js 22+ - **Language**: TypeScript - **Database**: SQLite (better-sqlite3) - **Token Counting**: tiktoken (GPT-4 tokenizer) - **Compression**: Brotli (built-in Node.js) - **Caching**: Multi-tier LRU/LFU/FIFO caching - **Protocol**: MCP SDK (@modelcontextprotocol/sdk) ### Supported AI Tools Token Optimizer works with stdio-capable MCP clients and includes first-party setup guidance for: - **OpenAI Codex** - Direct MCP setup or the bundled plugin, skill, and hooks - **Claude Code** - Direct MCP setup or the bundled plugin, skill, and hooks - **GitHub Copilot CLI** - MCP configuration, AGENTS.md guidance, and repository hooks - **Google Gemini CLI** - Direct MCP setup or the bundled context-and-hooks extension - **OpenCode** - Local MCP configuration, AGENTS.md instructions, and compaction plugin - **Claude Desktop**, **Cursor**, **Cline**, and **Windsurf** - Standard MCP JSON configuration See [Installation](#installation) for the supported commands and configuration files. ### Performance Characteristics - **Compression Ratio**: 2-4x typical (up to 82x for repetitive content) - **Context Window Savings**: 60-90% average across all operations - **Cache Hit Rate**: >80% in typical usage - **Operation Overhead**: <10ms for cache operations (optimized from 50-70ms) - **Compression Speed**: ~1ms per KB of text - **Hook Overhead**: <10ms per operation (7x improvement from in-memory optimizations) #### Performance Optimizations The PowerShell hooks have been optimized to reduce overhead from 50-70ms to <10ms through: - **In-Memory Session State**: Session data kept in memory instead of disk I/O on every operation - **Batched Log Writes**: Operation logs buffered and flushed every 5 seconds or 100 operations - **Lazy Persistence**: Disk writes only occur when necessary (session end, optimization, reports) #### Environment Variables Control hook behavior with these environment variables: The MCP server exposes an 18-tool core catalog by default so tool schemas do not consume a large share of the model context. Set `TOKEN_OPTIMIZER_TOOL_PROFILE=full` before starting the server to expose all 103 specialized tools. `TOKEN_OPTIMIZER_TOOL_PROFILE=core` is the explicit form of the default. Live graph capture uses `TOKEN_OPTIMIZER_TOOL_PROFILE=continuity`, which exposes only capture and query. The extended `cognitive` profile adds checkpoints, outcomes, and receipt attestation. The native-token audit measures 480 startup tokens for continuity, 694 for extended cognitive, 4,815 for core, and 30,125 for full. Stateful consumers normally receive zero MCP tools: host pre-action delivery adds only the selected capsule through the client lifecycle channel. Other enabled MCP servers add their own schemas independently. The current hardened cross-CLI smoke does not qualify. In the final Codex-to-Claude adversarial pair, both successors were correct and the runtime capsule was delivered with exact provider model attestations and zero consumer MCP tools, but runtime used 5.383% more tokens and 41.758% more latency. The reciprocal runtime arm was blocked by Claude provider quota, and Antigravity 1.1.11 required account authentication before a Gemini model could run. These signed outcomes keep the release verdict `insufficient`; they are evidence of working provenance and fail-closed behavior, not powered effectiveness or universal superiority. The replacement-grade protocol is intentionally larger than that smoke: 54,054 all-family/all-arm trial envelopes, 113,022 provider calls, three independent model families, direction-level non-inferiority, and 1,056 hard-negative opportunities per direction and arm so the Bonferroni-adjusted 95% family-wise false-delivery upper boundβ€”not only the point estimateβ€”must remain below 1%. Run `npm run verify:ucr:study-design` to validate the frozen metric coverage. The full protocol and CLI driver contract are in [`evals/ucr/FULL_STUDY_CONTRACT.md`](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/evals/ucr/FULL_STUDY_CONTRACT.md). #### Performance Controls - **`TOKEN_OPTIMIZER_USE_FILE_SESSION`** (default: `false`) - Set to `true` to revert to file-based session tracking (legacy mode) - Use if you encounter issues with in-memory session state - Example: `$env:TOKEN_OPTIMIZER_USE_FILE_SESSION = "true"` - **`TOKEN_OPTIMIZER_SYNC_LOG_WRITES`** (default: `false`) - Set to `true` to disable batched log writes - Forces immediate writes to disk (slower but more resilient) - Use for debugging or if logs are being lost - Example: `$env:TOKEN_OPTIMIZER_SYNC_LOG_WRITES = "true"` - **`TOKEN_OPTIMIZER_DEBUG_LOGGING`** (default: `true`) - Set to `false` to disable DEBUG-level logging - Reduces log file size and improves performance - INFO/WARN/ERROR logs still written - Example: `$env:TOKEN_OPTIMIZER_DEBUG_LOGGING = "false"` #### Development Path - **`TOKEN_OPTIMIZER_DEV_PATH`** - Path to local development installation - Automatically set to `~/source/repos/token-optimizer-mcp` if not specified - Override for custom development paths - Example: `$env:TOKEN_OPTIMIZER_DEV_PATH = "C:\dev\token-optimizer-mcp"` **Performance Impact**: Using in-memory mode (default) provides a 7x improvement in hook overhead: - Before: 50-70ms per hook operation - After: <10ms per hook operation - 85% reduction in hook latency ### Monitoring Token Savings #### Real-Time Session Monitoring **To view your actual token SAVINGS**, use the `get_session_stats` tool: ```typescript // View current session statistics with token savings breakdown await get_session_stats({}); ``` **Output includes:** - **Total tokens saved** (this is the actual savings amount!) - **Token reduction percentage** (e.g., "60% reduction") - **Cache hit rate** and **compression ratios** - **Breakdown by tool** (Read, Grep, Glob, etc.) - **Top 10 most optimized operations** with before/after comparison **Example Output:** ```json { "sessionId": "abc-123", "totalTokensSaved": 125430, // ← THIS is your savings! "tokenReductionPercent": 68.2, "originalTokens": 184000, "optimizedTokens": 58570, "cacheHitRate": 72.0, "byTool": { "smart_read": { "saved": 45000, "percent": 80 }, "smart_grep": { "saved": 32000, "percent": 75 } } } ``` #### Session Tracking Files All operations are automatically tracked in session data files: **Location**: `~/.claude-global/hooks/data/current-session.txt` **Format**: ```json { "sessionId": "abc-123", "sessionStart": "20251031-082211", "totalOperations": 1250, // ← Number of operations "totalTokens": 184000, // ← Cumulative token COUNT "lastOptimized": 1698765432, "savings": { // ← Auto-updated every 10 operations (Issue #113) "totalTokensSaved": 125430, // Tokens saved by compression "tokenReductionPercent": 68.2, // Percentage of tokens saved "originalTokens": 184000, // Original token count before optimization "optimizedTokens": 58570, // Token count after optimization "cacheHitRate": 42.5, // Cache hit rate percentage "compressionRatio": 0.32, // Compression efficiency (lower is better) "lastUpdated": "20251031-092500" // Last savings update timestamp } } ``` **New in v1.x**: The `savings` object is now automatically updated every 10 operations, eliminating the need to manually call `get_session_stats()` for real-time monitoring. This provides instant visibility into token optimization performance. **How it works**: - Every 10 operations, the PowerShell hooks automatically call `get_cache_stats()` MCP tool - Savings metrics are calculated from cache performance data (compression ratio, original vs compressed sizes) - The session file is atomically updated with the latest savings data - If the MCP call fails, the update is skipped gracefully without blocking operations **Note**: For detailed per-operation analysis, use `get_session_stats()`. The session file provides high-level aggregate metrics. #### Project-Wide Analysis Analyze token usage across your entire project: ```typescript // Analyze project token costs await analyze_project_tokens({ projectPath: '/path/to/project', }); ``` **Provides:** - Total token cost estimation - Largest files by token count - Optimization opportunities - Cost projections at current API rates #### Cache Performance Monitor cache hit rates and storage efficiency: ```typescript // View cache statistics await get_cache_stats({}); ``` **Metrics:** - Total entries - Cache hit rate (%) - Average compression ratio - Total storage saved - Most frequently accessed keys ### Troubleshooting #### CLI connection and discovery #### The tools do not appear in Codex 1. Run `codex mcp get token-optimizer`. 2. Confirm the entry is enabled with `codex mcp list`. 3. Start a new Codex conversation. 4. Run `/mcp` in the interactive CLI and inspect the server status. #### `codex` is blocked on Windows PowerShell may reject the `codex.ps1` shim under a restrictive execution policy. Use `codex.cmd` for the installation and verification commands, or review your user-scoped PowerShell execution policy. #### Remove Token Optimizer from Codex ```bash codex mcp remove token-optimizer ``` This removes the Codex registration; it does not delete your local cache or analytics database. #### Common Issues and Solutions #### Issue: "Invalid or malformed JSON" in Claude Code Settings **Symptom**: Claude Code shows "Invalid Settings" error after running install-hooks **Cause**: UTF-8 BOM (Byte Order Mark) was added to settings.json files **Solution**: Upgrade to v3.0.2+ which fixes the BOM issue: ```bash npm install -g @ooples/token-optimizer-mcp@latest ``` If you're already on v3.0.2+, manually remove the BOM: ```powershell # Windows: Remove BOM from settings.json $content = Get-Content "~/.claude/settings.json" -Raw $content = $content -replace '^\xEF\xBB\xBF', '' $content | Set-Content "~/.claude/settings.json" -Encoding utf8NoBOM ``` ```bash # Linux: Remove BOM from settings.json sed -i '1s/^\xEF\xBB\xBF//' ~/.claude/settings.json # macOS: Remove BOM from settings.json (BSD sed requires empty string after -i) sed -i '' '1s/^\xef\xbb\xbf//' ~/.claude/settings.json ``` #### Issue: Hooks Not Working After Installation **Symptom**: Token optimization not occurring automatically **Diagnosis**: 1. Check if hooks are installed: ```powershell # Windows Get-Content ~/.claude/settings.json | ConvertFrom-Json | Select-Object -ExpandProperty hooks ``` ```bash # macOS/Linux cat ~/.claude/settings.json | jq .hooks ``` 2. Verify dispatcher.ps1 exists: ```powershell # Windows Test-Path ~/.claude-global/hooks/dispatcher.ps1 ``` ```bash # macOS/Linux [ -f ~/.claude-global/hooks/dispatcher.sh ] && echo "Exists" || echo "Missing" ``` **Solution**: Re-run the installer: ```powershell # Windows powershell -ExecutionPolicy Bypass -File install-hooks.ps1 ``` ```bash # macOS/Linux bash install-hooks.sh ``` #### Issue: Low Cache Hit Rate (<50%) **Symptom**: Session stats show cache hit rate below 50% **Causes**: 1. Working with many new files (expected) 2. Cache was recently cleared 3. TTL (time-to-live) is too short **Solutions**: 1. **Warm up the cache** before starting work: ```typescript await cache_warmup({ paths: ['/path/to/frequently/used/files'], recursive: true, }); ``` 2. **Increase TTL** for stable APIs: ```typescript await smart_api_fetch({ url: 'https://api.example.com/data', ttl: 3600, // 1 hour instead of default 5 minutes }); ``` 3. **Check cache size limits**: ```typescript await smart_cache({ operation: 'configure', l1MaxSize: 2000, // Increase from default 1000 l2MaxSize: 20000, // Increase from default 10000 }); ``` #### Issue: High Memory Usage **Symptom**: Node.js process using excessive memory **Cause**: Large cache in memory (L1/L2 tiers) **Solution**: Configure cache limits: ```typescript await smart_cache({ operation: 'configure', evictionStrategy: 'LRU', // Least Recently Used l1MaxSize: 500, // Reduce L1 cache l2MaxSize: 5000, // Reduce L2 cache }); ``` Or clear the cache: ```typescript await clear_cache({}); ``` #### Issue: Slow First-Time Operations **Symptom**: Initial Read/Grep/Glob operations are slow **Cause**: Cache is empty, building indexes **Solution**: This is expected behavior. Subsequent operations will be 80-90% faster. To pre-warm the cache: ```typescript await cache_warmup({ paths: ['/src', '/tests', '/docs'], recursive: true, schedule: 'startup', // Auto-warm on every session start }); ``` #### Issue: "Permission denied" Errors on Windows **Symptom**: Cannot write to cache or log files **Cause**: PowerShell execution policy or file permissions **Solution**: 1. **Set execution policy**: ```powershell Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser ``` 2. **Check file permissions**: ```powershell icacls "$env:USERPROFILE\.token-optimizer" ``` 3. **Re-run installer as Administrator** if needed #### Issue: Cache Files Growing Too Large **Symptom**: `~/.token-optimizer/cache.db` is >1GB **Cause**: Caching very large files or many API responses **Solution**: 1. **Clear old entries**: ```typescript await clear_cache({ olderThan: 7 }); // Clear entries older than 7 days ``` 2. **Reduce cache retention**: ```typescript await smart_cache({ operation: 'configure', defaultTTL: 3600, // 1 hour instead of 7 days }); ``` 3. **Manually delete cache** (nuclear option): ```bash rm -rf ~/.token-optimizer/cache.db ``` #### Getting Help If you encounter issues not covered here: 1. **Check the hook logs**: `~/.claude-global/hooks/logs/dispatcher.log` 2. **Check session data**: `~/.claude-global/hooks/data/current-session.txt` 3. **File an issue**: [GitHub Issues](https://github.com/ooples/token-optimizer-mcp/issues) - Include debug logs - Include your OS and Node.js version - Include the output of `get_session_stats` ### Limitations - **Small Text**: Best for content >500 characters (cache overhead on small snippets) - **One-Time Content**: No benefit for content that won't be referenced again - **Cache Storage**: Automatic cleanup after 7 days to prevent disk usage issues - **Token Counting**: Uses GPT-4 tokenizer (approximation for Claude, but close enough) ## Development ```bash git clone https://github.com/ooples/token-optimizer-mcp.git cd token-optimizer-mcp npm ci npm run build npm test node scripts/mcp-smoke.mjs ``` To make Codex use your local build while developing: ```bash codex mcp add token-optimizer-local -- node /absolute/path/to/token-optimizer-mcp/dist/server/index.js ``` ## Documentation - [Quick start](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/QUICK_START_GUIDE.md) - [Tool reference](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/TOOLS.md) - [Codex and agent guidance](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/integrations/AGENTS.md) - [Hook installation](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/HOOKS-INSTALLATION.md) - [Testing](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/TESTING_INSTRUCTIONS.md) - [Contributing](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/docs/CONTRIBUTING.md) - [Security policy](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/SECURITY.md) - [Changelog](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/CHANGELOG.md) ## License MIT License - see [LICENSE](https://github.com/ooples/token-optimizer-mcp/blob/HEAD/LICENSE) for details ## Author Built for measurable token efficiency across supported coding agents by the ooples team.