Volante

A transparent, user-owned model router that checks its own answers (alpha). Volante routes a goal across the models you configured — enforcing hard capabilities first, ranking the rest on explainable metadata and evaluation evidence — and can then verify the result by RUNNING it: assertions derived from your goal, executed in a sandbox, reported check by check. Inventory, policy, credentials and decision traces stay on your machine. By default it answers in ONE call — across 72 paired runs, planning and synthesising scored -0.047 against that [95% CI -0.135, +0.041] at 8.8x the cost, so the interval bounds its best case near +0.04. --orchestrate opts back into the task DAG for the work that genuinely needs a tool loop, which one call cannot do.

What the measurements say, including the parts that do not flatter it. This project runs a 3-arm eval against itself and publishes the losses. Orchestration does not reliably beat a single strong model: an earlier release of this README claimed +0.289 at p<0.005, it failed to reproduce, and on the goal built to give decomposition headroom a stronger model in the same family scores 0.958 alone. Four predictive gates, a repair pass and a model-escalation policy were all built, measured, and discarded. What survived every test is narrower and real — the router, the reliability work (unparsable output 2 runs in 8 -> 0 in 8), and the verifier. Full numbers, method and corrections: eval/artifacts/README.md.

Selection quality is an evidence-based prediction, not a claim of a universal winner. Volante does not yet ship published representative cross-provider benchmarks or automatic score calibration. Built without an orchestration framework (no LangChain / CrewAI / LiteLLM).

A volante steers the game: the deep-lying midfielder who reads the whole pitch and sends the ball where it does the most good — then takes it back. One mind, many players.

Highlights

• Supervisor + explainable routing. An LLM plans a validated, acyclic task DAG; the router evaluates every configured model, rejects hard capability mismatches, scores the eligible set, and records the complete decision trace. Optional evaluation-derived quality profiles replace coarse metadata with evidence. quality, local, cheap, and cash_protect_quota are distinct routing objectives.
• User-owned, cross-provider control plane. AnthropicProvider and a generic OpenAICompatProvider speak to Anthropic, Google AI Studio (Gemini), Groq, OpenRouter, DeepSeek, Moonshot (Kimi), local Ollama, and any other OpenAI-compatible endpoint. The inventory and credentials stay in the user's environment.
• Scoped provider failover. Model/deployment, provider authentication/endpoint, exhausted-retry, and timeout failures are recorded and can move work to the next ranked eligible candidate, including planner/synthesizer fallbacks. Malformed or semantically invalid requests still fail fast instead of spraying the request across providers.
• Hybrid one-shot / agentic. Tasks run as a single call or as a model↔tool loop (run_python in a Docker-isolated sandbox when a daemon is available, withheld when none is — plus host-mediated fetch_url / read_file).
• Shared context. An append-only blackboard carries provenance; each task gets a scoped, budget-capped projection of only the dependency artifacts it needs.
• Streaming everywhere. Live token streaming through the supervisor, workers, and synthesizer, with per-task labels for parallel workers and cooperative early-stop.
• Optional Web UI. A small FastAPI + SSE app streams a run live in the browser (plan → per-task worker output → synthesis → result); runs with real providers or a no-key demo.
• Answers you can check. volante --verify derives assert statements from your goal, runs them against the result in the sandbox, and prints which passed and which failed — with the check text, not a badge, because a failing check is often the CHECK being wrong and only the line shows that. It reports three states, including "not enough evidence" when your goal states no expected result. Measured: 0 false positives and 0 false negatives across 8 goals against a coarse defect, 5 of 6 caught against edit-shaped ones. A clean report is evidence, not a guarantee.
• Cost & honesty. A CostMeter tallies per-model usage and cost, and propagates an estimated flag when a provider returns no usage.
• Forgery-resistant evaluation that reports its own power. A 3-arm eval (baseline vs. orchestration vs. single-agent) with a scorer that runs untrusted solution code under process + filesystem separation so a model cannot fake a passing score. Arms are compared PAIRWISE and every report prints a confidence interval and the smallest effect the run could have detected — added after three published numbers had to be corrected downward, all of them point estimates from designs too small to support them.
• Tested. 1200+ tests, zero-network by default (FakeProvider + local subprocesses), ruff-clean, mypy-clean.

Architecture

flowchart TD
    goal(["goal"]) --> S["Supervisor<br/>plan → validated task DAG<br/>(acyclic · typed · one_shot | agentic)"]
    S --> R["Router<br/>best predicted fit per task<br/>(hard constraints → explainable score)"]
    R --> P
    subgraph wave["wave execution · asyncio fan-out · fail-fast"]
        direction TB
        P["Projector<br/>scoped, budget-capped request<br/>(system + task + deps)"]
        P --> W["Worker<br/>one-shot"]
        P --> AW["AgenticWorker<br/>model ↔ tool loop<br/>(run_python · fetch_url · read_file)"]
    end
    W --> BB[("Blackboard<br/>append-only · provenance · latest-wins")]
    AW --> BB
    BB --> SY["Synthesizer<br/>combine artifacts → final answer"]
    SY --> result(["result<br/>+ CostMeter totals · usage · duration"])

    classDef io stroke:#8b5cf6,stroke-width:2px;
    classDef store stroke:#f59e0b,stroke-width:2px;
    class goal,result io;
    class BB store;

                    ┌──────────────┐
   goal ──────────► │  Supervisor  │  plan → validated task DAG (acyclic, typed, one_shot|agentic)
                    └──────┬───────┘
                           ▼
                    ┌──────────────┐   per task: hard-capability filter, then rank the
                    │    Router    │   predicted fit from configured evidence
                    └──────┬───────┘
                           ▼
        ┌───────────── wave execution (asyncio, fan-out cap, fail-fast) ─────────────┐
        │   ┌───────────┐   scoped, budget-capped request (system + task + deps)      │
        │   │ Projector │──────────────────────────────────────────────────────────► │
        │   └───────────┘                                                             │
        │        ▼                              ▼                                      │
        │   ┌─────────┐  one-shot          ┌───────────────┐  model↔tool loop         │
        │   │ Worker  │                    │ AgenticWorker │  (run_python sandbox,     │
        │   └────┬────┘                    └───────┬───────┘   fetch_url, read_file)   │
        │        └──────────────┬──────────────────┘                                   │
        └───────────────────────┼───────────────────────────────────────────────────┘
                                 ▼
                    ┌──────────────────────┐   append-only, provenance, latest-wins
                    │  Blackboard          │◄──────────────────────────────────────
                    └──────────┬───────────┘
                               ▼
                    ┌──────────────┐
                    │ Synthesizer  │  combine artifacts → final answer
                    └──────┬───────┘
                           ▼
                        result  (+ CostMeter totals, usage, duration)

Quickstart

Requires Python 3.11.10+ (the floor is a security boundary — see Providers).

Install it: