GoModel

What GoModel does

GoModel acts as a self-hosted gateway for MCP servers and hosted or self-hosted AI providers. Its MCP gateway consolidates upstream MCP servers behind one authenticated endpoint, so an MCP client can connect to a central service instead of managing each backend separately.

The broader gateway also exposes OpenAI-compatible endpoints under /v1 and an Anthropic-compatible endpoint at /v1/messages. This allows the official OpenAI and Anthropic SDKs to use GoModel by changing their base URL. Supported provider categories include commercial APIs, OpenAI-compatible services, and local or self-hosted systems such as Ollama, SGLang, vLLM, and llm-d.

How it works

GoModel receives requests at its gateway, applies configured access and routing rules, and forwards them to an appropriate provider or upstream MCP server. The service can use virtual model names with aliases and round-robin or cost-based load balancing. Failover, retries, and circuit breakers can reroute requests when a provider is unavailable.

For AI inference, the gateway includes request caching, cost estimates, usage analytics, budgets, rate limits, session pinning, model access controls, request labels, and hierarchical user paths. These controls can apply across users, teams, keys, providers, models, and request paths. The dashboard provides administration and visibility into usage and spending.

Setup and configuration

A Docker installation publishes port 8080 and stores application data in a named volume:

docker run --rm -p 8080:8080 \
  -v gomodel_data:/app/data \
  enterpilot/gomodel

macOS and Linux users can install with the published shell installer, while Windows users can use the PowerShell installer. After starting the service, open the dashboard at http://localhost:8080/admin/dashboard to add a provider and manage key settings.

Configuration can be supplied through .env, config.yaml, or exported environment variables. Later sources override earlier ones in this order: built-in defaults, config.yaml, .env, then exported environment variables. Docker Compose options are available for an infrastructure stack with Redis, PostgreSQL, MongoDB, and Adminer, or for a full stack that also builds GoModel and adds Prometheus.

Tools and capabilities

• Aggregate MCP servers behind one authenticated HTTP endpoint.
• Route requests across many hosted, local, and OpenAI-compatible providers.
• Serve OpenAI-compatible and Anthropic-compatible request formats.
• Cache exact or semantically equivalent responses.
• Enforce budgets, request limits, token limits, concurrency limits, and model allowlists.
• Track estimated costs, usage, labels, and audit information.
• Provide provider-native passthrough routes under /p/{provider}/....
• Expose OpenAI-compatible text-to-speech, transcription, image generation, and image editing APIs.
• Preserve Gemini thought signatures and Anthropic thinking blocks across supported turns and provider paths.

Limitations and notes

GoModel requires a running gateway instance and configured access to the providers or MCP servers it should expose. Provider behavior and supported features vary by backend; the project maintains a provider feature matrix for those differences. The default local service listens on port 8080, and the dashboard is used to configure providers after startup.