Llm Router

What ypollak2/llm-router MCP server does

ypollak2/llm-router MCP server sits in the lifecycle of supported coding tools and selects a model for each prompt. Routine requests can be sent to local or lower-cost models, while more demanding work can remain on premium models. The goal is to reduce use of a Claude subscription quota without requiring a separate hosted proxy.

The router supports more than 20 providers, including OpenAI, Gemini, Groq, Ollama, and Codex. Its routing decisions consider task complexity, the current pressure on a Claude subscription, configured providers, budget settings, and the selected routing policy. Example outcomes include sending an error explanation to Ollama or Gemini Flash, an endpoint refactor to GPT-4o or Gemini Pro, and a distributed tracing design to o3 or Claude Opus.

How it works

For Claude Code, automatic routing is implemented through six hooks that inspect prompts before the model handles them. The project also exposes 30 MCP tools for routing and related operations. Codex CLI uses manual MCP tools in the documented setup, while Gemini CLI supports automatic routing through hooks.

The project is local-first: it does not require a hosted proxy or an account. On Claude Pro or Max, it can work with no provider API keys by using MCP tools and local models. Optional credentials make additional providers available, including OpenAI, Gemini, OpenRouter, and local Ollama endpoints. A semantic deduplication cache and prompt caching can avoid repeated work, while a daily spend cap provides a budget boundary. A live web dashboard is included for viewing router activity and status.

Setup and configuration

Install the Python package named llm-routing, then run its llm-router install command. The project requires Python 3.11 or newer. Provider configuration is optional for Claude subscription users, but environment variables can be supplied when external or local providers are needed:

• OPENAI_API_KEY for OpenAI models
• GEMINI_API_KEY for Gemini models
• OLLAMA_BASE_URL for an Ollama service
• OPENROUTER_API_KEY for OpenRouter models

Use llm-router health to check provider connectivity after setup. The cost_aggressive policy uses the OpenRouter workhorse pool when an OpenRouter key is configured.

Tools and capabilities

ypollak2/llm-router MCP server provides:

• Complexity-aware model selection
• Subscription-quota protection for Claude Code
• Routing across free, budget, and premium model tiers
• Six automatic routing hooks
• Thirty MCP tools
• Semantic duplicate-request caching
• Prompt caching
• A daily spending limit
• A live web dashboard
• Health checks for configured providers

Limitations and notes

Routing depends on the providers configured on the host and on the selected policy, so the exact model chain can vary. External providers may require their own API keys; the zero-key workflow applies to Claude Pro or Max usage with local models and MCP-based routing. Ollama also requires a reachable local service.

The repository documents RouterArena results and notes that some classification approaches did not outperform simpler model selection. Leaderboard position can change, and reported savings are host- and client-dependent rather than universal guarantees. Codex support is described as manual MCP tooling, with automatic hooks listed as planned rather than available.