Multi-LLM Provider for Go

multi-llm-provider-go is a Go library for using hosted LLM APIs and local coding agents through a shared set of provider interfaces.

It supports two complementary ways to run a model:

• API providers call hosted models through their normal SDK or HTTP transport, including OpenAI, Anthropic, Bedrock, Vertex AI, Azure, and OpenRouter.
• Coding-agent providers run Claude Code, Codex CLI, Cursor Agent, or Pi in local tmux sessions, preserving their native tools, subscriptions, project context, and authenticated sessions.

The repository also includes llm-provider-mcp, an optional MCP server for delegating asynchronous work between coding agents. It is one way to expose the provider library—not the library's only use case.

Why This Exists

Applications should be able to choose the right model and transport for each task without rebuilding their orchestration layer.

Use a direct API when you want a conventional request/response integration, predictable infrastructure, or model-level features such as structured output, embeddings, and media generation. Use a coding-agent CLI when you want an agent that can inspect a repository, edit files, run commands, and reuse an existing local subscription. Both fit behind the same Go model abstraction.

flowchart LR
    App[Go application] --> Provider[Shared provider interfaces]
    Provider --> API[API and cloud adapters]
    Provider --> CLI[Local coding-agent adapters]
    API --> Hosted[Hosted models]
    CLI --> Tmux[tmux sessions]
    Tmux --> Agents[Claude Code / Codex / Cursor / Pi]
    MCP[Optional MCP server] --> CLI

Supported Providers

The core InitializeLLM factory returns the same llmtypes.Model interface for text and coding-agent providers:

Specialized factories expose capabilities that do not fit the text-model interface:

Gemini models are available through Vertex AI for direct API access or through Pi as a coding agent. The old Gemini CLI adapter has been removed.

Common Capabilities

Provider support varies, but the shared interfaces cover:

• Text generation and streaming
• Tool calling and structured output
• Token usage, model metadata, logging, and event emission
• Embeddings
• Image input and generation
• Video generation and conversational video editing
• Audio generation and transcription
• Music generation
• Stateful coding-agent sessions, continuation, and terminal progress

Quick Start: Go Library

Install the module:

go get github.com/manishiitg/multi-llm-provider-go@latest

The current module requires Go 1.25.12 or newer.

Initialize a provider and use the returned llmtypes.Model:

package main

import (
    "context"
    "fmt"
    "log"

    llmproviders "github.com/manishiitg/multi-llm-provider-go"
    "github.com/manishiitg/multi-llm-provider-go/llmtypes"
)

func main() {
    model, err := llmproviders.InitializeLLM(llmproviders.Config{
        Provider: llmproviders.ProviderOpenAI,
        ModelID:  "gpt-4.1-mini",
    })
    if err != nil {
        log.Fatal(err)
    }

    response, err := model.GenerateContent(
        context.Background(),
        []llmtypes.MessageContent{
            llmtypes.TextParts(llmtypes.ChatMessageTypeHuman, "Explain tmux in one sentence."),
        },
    )
    if err != nil {
        log.Fatal(err)
    }

    if len(response.Choices) == 0 {
        log.Fatal("provider returned no choices")
    }
    fmt.Println(response.Choices[0].Content)
}

Set the credential expected by the selected provider—for example, OPENAI_API_KEY for OpenAI. Credentials can also be supplied explicitly with Config.APIKeys. See the examples for streaming, tool calls, custom logging, Bedrock, and Vertex AI.

Configuration

llmproviders.Config controls provider initialization:

Common credential sources include:

See .env.example for common provider credentials. Model, endpoint, fallback, and test-specific variables are documented next to the provider adapters and tests that consume them.

Changing transports starts with changing the provider:

config.Provider = llmproviders.ProviderAnthropic  // direct API
config.Provider = llmproviders.ProviderBedrock    // cloud API
config.Provider = llmproviders.ProviderCodexCLI   // local coding agent
config.Provider = llmproviders.ProviderCursorCLI  // local coding agent

Bounded coding-agent calls can use the process working directory and a temporary session automatically. Long-lived host applications should explicitly pass CodingAgentWorkingDirOption, CodingAgentInteractiveSessionOption, and CodingAgentPersistentInteractiveOption. Provider-specific options additionally control the model, approval policy, resume ID, tools, and streaming behavior.

Coding Agents And tmux

The coding-agent adapters turn native coding CLIs into providers without reimplementing their agent loops. They use each CLI's existing login and model access, and run in a local project with that CLI's native file and shell tools.

tmux is the default transport because it supports long-lived interactive sessions, multi-turn continuation, live terminal capture, control-key input, and recovery after a caller disconnects.

Requirements for this transport:

• macOS or Linux
• tmux 3.x or newer
• At least one installed and authenticated coding CLI

The library exposes session lifecycle, resume, input, interrupt, pane capture, and cleanup helpers so a host application can manage coding agents as part of its own workflow.

Module Layout

multi-llm-provider-go/
├── providers.go                 # Provider IDs, configuration, initialization
├── provider_*.go                # Shared provider behavior and media factories
├── interfaces/                  # Logging, events, and public support contracts
├── llmtypes/                    # Messages, responses, tools, streams, options
├── pkg/adapters/                # API, cloud, media, and coding-CLI adapters
├── pkg/codingagentjob/          # Durable asynchronous job execution
├── pkg/codingagentmcp/          # Optional MCP tool surface
├── pkg/tmuxcapture/             # Terminal progress capture and cleanup
├── cmd/llm-chat/                # Local provider chat client
├── cmd/llm-test/                # Manual provider contract runner
└── cmd/llm-provider-mcp/        # Optional delegation MCP server

llmtypes.Model is the central text request/response interface. Additional interfaces cover embeddings, image generation, video generation, audio generation and transcription, and music generation.

Optional: Asynchronous Delegation Over MCP

llm-provider-mcp packages the coding-agent providers as a local stdio MCP server. A Codex or Claude Code host can queue work in another coding CLI, continue working, and retrieve the result later. Jobs are persisted in SQLite and executed in detached tmux sessions.

Install it in the project where you want delegation: