Llm Router

What ypollak2/llm-router does

ypollak2/llm-router routes coding prompts between Claude subscription usage, local models, free providers, budget models, and premium models. Its main use case is protecting a Claude Pro or Max five-hour quota from routine requests that can be handled by a cheaper or local model. It can also route paid API traffic according to task complexity and budget.

The project connects to more than 20 providers, including OpenAI, Gemini, Groq, Ollama, and Codex. Provider API keys are optional for the Claude subscription workflow; adding keys makes more providers available. Ollama is also used for loopback-only handling of routed prompts that contain detected secrets.

How it works

Lifecycle hooks inspect a prompt before the host model processes it. A regular-expression heuristic classifies some prompts immediately; prompts that cannot be classified use a default route. Routable prompts begin with a draft from a free or local model and move toward paid models only when required.

Drafts are advisory by default. Claude still handles the turn unless LLM_ROUTER_ZERO_CLAUDE=1 is set. Tool-call enforcement defaults to smart, which can hold selected tool calls until routing completes; LLM_ROUTER_ENFORCE=off disables that behavior.

Provider failures and rate limits trigger circuit breakers, allowing the router to skip an unhealthy provider instead of repeatedly retrying it. The router also supports semantic deduplication, prompt caching, a daily spend cap, media requests, and multi-step orchestration. Session summaries report savings against a baseline, tier mix, provider cost, latency percentiles, and top routes.

Setup and configuration

Install the Python package and command-line entry point with:

pip install llm-routing
llm-router install
llm-router doctor
llm-router status

llm-router install targets Claude Code by default. Host-specific installation is available for Codex CLI, OpenCode, Gemini CLI, GitHub Copilot CLI, OpenClaw, Trae, Factory Droid, Claude Desktop, VS Code, Cursor, GitHub Copilot in VS Code, and Windsurf/Cascade. llm-router install --host all processes every supported host configuration in one pass.

The optional daily KPI job is installed only when requested or accepted during installation, and uninstall removes it. Python agents can use the router in process with Router(budget_usd=0.50).run(...); asynchronous code can call .arun(...), and the budget raises BudgetExceeded after the cap is spent.

Tools and capabilities

The default MCP surface exposes 12 front-door tools; setting LLM_ROUTER_SLIM=off exposes all registered tools. The broader installation provides 30 MCP tools, including:

• llm_image, llm_video, and llm_audio for media requests.
• llm_orchestrate for multi-step research workflows.
• Routing, provider health, status, and diagnostic operations through the MCP and CLI surfaces.
• Status output, terminal titles, and operating-system notifications showing the last routed model, savings, and provider health.

Secret-bearing routed calls are restricted to local Ollama handling or refused, unless a request explicitly uses the documented cloud: override. This protection applies to the router's text tools, gateway, and streaming paths, not to prompts already sent by a host to its own model.

Limitations and notes

Routed responses do not replace Claude turns unless LLM_ROUTER_ZERO_CLAUDE=1 is configured. Host capabilities differ: GitHub Copilot in VS Code is supported as a capability extension but does not provide cost routing. Pi and Kimi Code installation targets are planned and currently report an explanation rather than installing.

The prompt classifier is heuristic and only classifies a portion of measured prompts with confidence; the remaining prompts use a default route. Savings depend on the workload, and the status command separates verified and unverified savings rather than presenting one combined figure.