smigolsmigol/llmkit

Know what your AI agents cost.

Open-source API gateway for AI providers. Logs every request with token counts and dollar costs.
Budget limits reject requests before they reach the provider, not after.

$ npx @f3d1/llmkit-cli -- python my_agent.py

  $0.0215 total  3 requests  4.2s  ~$18.43/hr

  claude-sonnet-4-20250514  1 req    $0.0156  ████████████████████
  gpt-4o                    2 reqs   $0.0059  ███████░░░░░░░░░░░░░

Works with Python, Ruby, Go, Rust - anything that calls the OpenAI or Anthropic API. One command, no code changes.

Get started

1. Create an account at llmkit.sh (free while in beta)
2. Create an API key in the Keys tab
3. Pick a method below

CLI

Wrap any command. The CLI intercepts API calls, forwards them through the proxy, and prints a cost summary when the process exits.

npx @f3d1/llmkit-cli -- python my_agent.py

Use -v for per-request costs as they happen, --json for machine-readable output.

Python

pip install llmkit-sdk

With the proxy (budget enforcement, logging, dashboard):

from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmkit.sh/v1",
    api_key="llmk_your_key_here",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "hello"}],
)

Without the proxy (local cost estimation, zero setup):

from llmkit import tracked
from openai import OpenAI

client = OpenAI(http_client=tracked())

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "hello"}],
)
# costs estimated locally from bundled pricing table

tracked() wraps your HTTP client and estimates costs from token usage. No proxy needed. Works with any SDK that accepts http_client.

Framework integrations (LangChain, LlamaIndex, Pydantic AI):

from llmkit.integrations.langchain import LLMKitCallbackHandler
handler = LLMKitCallbackHandler()
chain.invoke("...", config={"callbacks": [handler]})
print(f"${handler.total_cost:.4f}")

TypeScript

npm install @f3d1/llmkit-sdk

import { LLMKit } from '@f3d1/llmkit-sdk'

const kit = new LLMKit({ apiKey: process.env.LLMKIT_KEY })
const agent = kit.session()

const res = await agent.chat({
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  messages: [{ role: 'user', content: 'summarize this document' }],
})

console.log(res.content)
console.log(res.cost)   // { inputCost: 0.003, outputCost: 0.015, totalCost: 0.018, currency: 'USD' }

Streaming, CostTracker, and Vercel AI SDK provider also available.

MCP Server

Query AI costs from Claude Code, Cline, or Cursor:

{
  "mcpServers": {
    "llmkit": {
      "command": "npx",
      "args": ["@f3d1/llmkit-mcp-server"],
      "env": { "LLMKIT_API_KEY": "llmk_your_key_here" }
    }
  }
}

11 tools - 6 proxy (need API key), 5 local (no key, auto-detect Claude Code + Cline + Cursor):

llmkit_usage_stats llmkit_cost_query llmkit_budget_status llmkit_session_summary llmkit_list_keys llmkit_health llmkit_local_session llmkit_local_projects llmkit_local_cache llmkit_local_forecast llmkit_local_agents

SessionEnd hook - auto-log session costs when Claude Code exits. Add to settings.json:

{
  "hooks": {
    "SessionEnd": [
      {
        "type": "command",
        "command": "npx @f3d1/llmkit-mcp-server --hook"
      }
    ]
  }
}

Parses the session transcript and prints cost summary. No API key needed.

GitHub Action

Cap AI spend in CI. The action runs your command through the CLI, tracks cost, and fails the job if it exceeds the budget.

- uses: smigolsmigol/llmkit/.github/actions/llmkit-budget@main
  with:
    command: python agent.py
    budget-usd: '5.00'
    post-comment: 'true'

Posts a cost report as a PR comment. Outputs total-cost, total-requests, budget-exceeded, and summary-json for downstream steps.

Why LLMKit

Most cost tracking tools give you "soft limits" that agents blow past in the first hour. LLMKit runs cost estimation before every request. If it would exceed the budget, the request gets rejected before reaching the provider. Per-key or per-session scope.

Tag requests with a session ID or end-user ID to track costs per agent, per conversation, per user. The dashboard and MCP server surface this data in real time. Cost anomaly detection alerts when a single request costs 3x the recent median.

11 providers through one interface: Anthropic, OpenAI, Google Gemini, Groq, Together, Fireworks, DeepSeek, Mistral, xAI, Ollama, OpenRouter. Fallback chains with one header (x-llmkit-fallback: anthropic,openai,gemini).

Runs on Cloudflare Workers at the edge. Cache-aware pricing across 7 providers with prompt caching. 730+ models priced across all providers.

Automatic prompt caching for Anthropic: the proxy injects cache breakpoints on system prompts and conversation history. Second request with the same system prompt costs 90% less. Zero config, zero code changes.

Framework integrations: drop-in cost tracking for LangChain, LlamaIndex, and Pydantic AI via callback handlers. Works alongside the httpx transport for direct SDK use.

470+ tests, ClusterFuzzLite fuzzing, 6-stage security pipeline (gitleaks, semgrep, CodeQL, bandit, pip-audit, pnpm audit). OpenSSF Scorecard 8.3 - higher than React, Django, Kubernetes, and every AI gateway competitor.

Public API endpoints (no auth required):

• /v1/pricing/compare - compare cost across all 730+ models for a given token count

Security

LLMKit handles your API keys. We take that seriously.

Full details in SECURITY.md.

git clone https://github.com/smigolsmigol/llmkit
cd llmkit && pnpm install && pnpm build

cd packages/proxy
echo 'DEV_MODE=true' > .dev.vars
pnpm dev
# proxy running at http://localhost:8787

npx wrangler login
npx wrangler secret put SUPABASE_URL
npx wrangler secret put SUPABASE_KEY
npx wrangler secret put ENCRYPTION_KEY
npx wrangler deploy

Star this repo if you find it useful.