AISIX AI Gateway

The open-source, Rust-native AI gateway for LLMs and AI agents

One OpenAI-compatible API in front of every model. Route, govern, secure, cache, and observe all your LLM and AI-agent traffic from a single control point — shipped as one static binary with low per-request overhead. Run it in your infrastructure for free, forever.

Built by the original creators of Apache APISIX.

Start free · Documentation · Quickstart · AISIX Cloud · Roadmap

AISIX AI Gateway is a Rust-native gateway that puts a single, OpenAI-compatible API in front of every LLM provider — OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, DeepSeek, and any OpenAI-compatible endpoint. It gives platform teams one place to route, govern, secure, and observe LLM traffic, with first-class SSE streaming and low gateway overhead.

It runs as a single static binary — low cold-start, lock-free config reads, and hot configuration reloads with no restarts: declare resources in one resources.yaml and reload on SIGHUP, or point the gateway at etcd for a multi-replica cluster. Run the open-source gateway in your infrastructure, or connect it to AISIX Cloud for centralized management with team governance, budgets, audit, and a dashboard.

AISIX AI Gateway (this repo) is the open-source product. It runs without a control plane using declarative configuration or etcd. When connected to AISIX Cloud, the same gateway serves as the data plane. AISIX Cloud adds a commercial control plane, either hosted by API7 (Hybrid Cloud) or hosted by you in your infrastructure (On-Premises). In both options, the gateway runs in your environment and calls providers directly; live AI traffic does not pass through the control plane or API7. The proxy API is identical throughout. Talk to us about AISIX Cloud →

⚡ Quickstart

One container. No control plane, no database, no configuration store — the gateway reads every dynamic resource from one declarative resources.yaml.

# config.yaml
resources_file: /etc/aisix/resources.yaml
proxy:
  addr: "0.0.0.0:3000"
admin:
  enabled: false          # a declarative gateway needs no admin listener
observability:
  metrics:
    prometheus:
      enabled: true
      addr: "0.0.0.0:9090"

# resources.yaml
_format_version: "1"

provider_keys:
  - display_name: openai-main
    provider: openai
    api_key: ${OPENAI_API_KEY}        # interpolated from the environment

models:
  - display_name: my-model
    provider: openai
    model_name: gpt-4o-mini
    provider_key: openai-main

api_keys:
  - display_name: local-dev
    key_env: CALLER_API_KEY           # hashed at load; the plaintext is never stored
    allowed_models: ["my-model"]

export OPENAI_API_KEY="YOUR_PROVIDER_KEY"
export CALLER_API_KEY="YOUR_CALLER_KEY"

docker run -d --name aisix \
  --platform linux/amd64 \
  -v "$(pwd)/config.yaml:/etc/aisix/config.yaml:ro" \
  -v "$(pwd)/resources.yaml:/etc/aisix/resources.yaml:ro" \
  -e OPENAI_API_KEY -e CALLER_API_KEY \
  -p 3000:3000 -p 127.0.0.1:9090:9090 \
  ghcr.io/api7/aisix:latest        # proxy → :3000, metrics + status → :9090
#                                  ^ the metrics/status listener is unauthenticated;
#                                    keep it on loopback or a private network

Then call the gateway exactly like OpenAI:

curl http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer $CALLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'

Edit resources.yaml and send SIGHUP (docker kill -s HUP aisix) to apply changes with no restart — an invalid file is rejected whole and the last good configuration keeps serving. Check a file before booting with aisix validate --resources resources.yaml.

Full walkthrough: the Gateway Quickstart · every field: the resources file reference. For a multi-replica cluster, point the gateway at etcd instead — resources_file and etcd are mutually exclusive.

✨ Why AISIX

• One API, every model. Speak the OpenAI or Anthropic wire format in; the gateway translates to whichever provider each model points at. Point an OpenAI or Claude SDK at one base_url and switch models without changing code.
• A real gateway, in Rust. Single static binary, low cold-start, lock-free config reads on the hot path, native streaming.
• Open source, free forever. Apache-2.0 licensed and built to run in your infrastructure. Choose AISIX Cloud when you want centralized management through a control plane and dashboard.
• Production controls built in. Routing & failover, rate limits, guardrails, caching, and observability ship in the box. (Budgets and spend caps are an AISIX Cloud feature — the gateway enforces the control plane's decisions.)

🧩 Features — available today

Covered by 183 end-to-end scenario files (496 cases) that run against real gateway processes.