GoModel - The last AI gateway you will ever need

GoModel is the fastest and the most resource-efficient AI Gateway (the self-reproducible benchmarks). It's an alternative to LiteLLM (which was hacked recently) and Portkey (which is no longer maintained on GitHub).

(click on the animation ↑ to see the live demo)

GoModel saves you money and nerves.

Money - because you can remember the responses on this layer (caching), track your spending and do tricks like prompt compression and intelligent routing.

Nerves - because we strive to achieve good quality and reliability. Our ambition is to be the last AI gateway you will need - the most reliable, resource-optimal, feature-rich and fast.

Quick Start

Step 1: Install and start GoModel

macOS / Linux

curl -fsSL https://gomodel.enterpilot.io/install.sh | sh
# OPENAI_API_KEY="your-openai-key" # (optional)
gomodel

Windows (PowerShell)

irm https://gomodel.enterpilot.io/install.ps1 | iex
# $env:OPENAI_API_KEY = "your-openai-key" # (optional)
gomodel

Docker

docker run --rm -p 8080:8080 \
  -e OPENAI_API_KEY="your-openai-key" \
  enterpilot/gomodel

ℹ️ You can configure GoModel with .env variables, a config.yaml file, OR directly in the dashboard.

ℹ️ Full list of environment variables (including all available providers): .env.template

ℹ️ The most secure way in production is to use .env to load API keys.

Step 2: Open the dashboard

http://localhost:8080/admin/dashboard

Step 3: Make your first API call

curl http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5-chat-latest",
    "input": "Hello!"
  }'

Using GoModel with official SDKs

GoModel exposes an OpenAI-compatible API at /v1 and an Anthropic-compatible API at /v1/messages, so the official SDKs work unchanged - just point the base URL at your GoModel server and use your GoModel key (set up with the GOMODEL_MASTER_KEY env variable or one generated in the dashboard).

OpenAI SDK

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",  # your GoModel server
    api_key="your-gomodel-key",
)

TypeScript / JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080/v1", // your GoModel server
  apiKey: "your-gomodel-key",
});

Anthropic SDK

The Anthropic SDK authenticates with x-api-key, which GoModel accepts alongside Authorization: Bearer.

Python

from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:8080",  # your GoModel server (no /v1 suffix)
    api_key="your-gomodel-key",
)

TypeScript / JavaScript

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "http://localhost:8080", // your GoModel server (no /v1 suffix)
  apiKey: "your-gomodel-key",
});

Supported LLM Providers

GoModel supports OpenAI, Anthropic, Cohere, Google Gemini, Vertex AI, DeepSeek, Groq, Fireworks AI, Meta (Muse Spark), OpenRouter, Z.ai, xAI (Grok), Alibaba Cloud Model Studio (Bailian), Kilo AI, MiniMax, Xiaomi MiMo, OpenCode Go, Azure OpenAI, Oracle, Ollama, SGLang, vLLM, llm-d, Amazon Bedrock Runtime, Amazon Bedrock Mantle, and all OpenAI-compatible providers. Voice: ElevenLabs (text-to-speech and speech-to-text).

See the Providers Overview for the full per-provider feature matrix (chat, /responses, embeddings, files, batches, passthrough), credentials, and configuration notes.

Docker Compose

Infrastructure only (Redis, PostgreSQL, MongoDB, Adminer - no image build):

cp .env.template .env
# Add your API keys to .env
docker compose up -d
# or: make infra

Full stack (adds GoModel + Prometheus; builds the app image):

docker compose --profile app up -d
# or: make image

Building the Docker Image Locally

docker build -t gomodel .
docker run --rm -p 8080:8080 --env-file .env gomodel

API Endpoints

GoModel exposes OpenAI-compatible and Anthropic-compatible APIs, provider-native passthrough, and operations routes. See the API Endpoints reference for the full endpoint tables, and Admin Endpoints for the admin REST API and dashboard.

Gateway Configuration

GoModel is configured through environment variables and an optional config.yaml. Environment variables override YAML values. See the Configuration reference for the full list of settings organized by category, along with .env.template and config/config.example.yaml.

Features

• Caching - exact and semantic response caching, so repeated prompts cost nothing
• Cost tracking - per-request cost estimates, usage analytics, and spending breakdowns in the dashboard
• Budgets - hard spend limits per user, team, or key
• Rate limits - requests, tokens, and concurrency caps per user path, provider, or model
• Virtual models - aliases and load balancing (round-robin or cost-based) behind stable model names
• Failover - automatic rerouting to backup providers, with retries and circuit breakers
• Labelling - tag requests from HTTP headers or API keys and break down usage by label
• User paths - hierarchical scoping of keys, model access, budgets, usage, and audit logs
• MCP gateway - aggregate your MCP servers behind one authenticated endpoint
• Passthrough API - provider-native APIs under /p/{provider}/..., with GoModel auth and tracking
• Guardrails - request and response policies enforced at the gateway
• Provider key rotation - round-robin over multiple API keys to lift per-key rate limits
• Observability - Prometheus metrics, audit logs, and live request streaming in the dashboard

Roadmap

See the Roadmap for commercial features and the public 0.2.0 milestone.

Sponsors

Community

Join our Discord to connect with other GoModel users.