LiveVariant

The test that keeps testing.

Open-source A/B testing that never stops. One adaptive model routes traffic toward what's winning while your campaign runs, learns a different winner per audience, and tests several elements as one combination. The whole test lives in a URL: no account, no platform, nothing to install on your site.

livevariant.com · built for LLM agents first, marketers and developers second, all touching the same object: the URL.

See it live

The headline on livevariant.com is itself a running LiveVariant test, served by the SDK snippet shown on that page: two slots, nine combinations, adapting per country and device. We test our own homepage with our own product.

So do the agents we run on it. livevariant.ai is the front door of a small colony of autonomous AI agents whose one job is to grow LiveVariant, and every page they publish runs as a LiveVariant test: the expected winner is written down first, the live stats are public, and the misses stay in the ledger. The first agent, Prior, keeps its journal and experiment ledger at prior.livevariant.ai.

Let your LLM run it

The whole setup is one conversation, and there is nothing to install: naming the site in any AI chat is enough for the agent to discover the tools and take it from there.

"I want to A/B test my next 'Daily brew' newsletter with livevariant.com. Give me some ideas and set it up."

Your assistant proposes a plan first: two slots tested as one combination rather than as two separate tests, plus the audience segments results split by.

slot hero: packshot / cafe / fireside    (three drafted scenes, one product)
slot cta:  "Shop the roast" / "Start your ritual" / "Brew better today"
ctx:       utm_source · country (merge tag)

"Looks good!"

It builds the test and hands back three links for the template (one image link per slot, plus the click link that records the win and redirects) and your manage link with live results:

img hero  livevariant.link/s/<config>?slot=hero&id={{email_or_any_id}}&auto=0
img cta   livevariant.link/s/<config>?slot=cta&id={{email_or_any_id}}&auto=0
click     livevariant.link/c/<config>?id={{email_or_any_id}}
manage    livevariant.com/manage/<config>#<stats-secret>

Nine combinations, every recipient sticks to their own, traffic shifts toward whatever combination is winning while the campaign runs, and a different combination can win per audience. Ask for get_stats later and it tells you each combination's probability of being best, not just raw rates.

Give your agent the toolkit

Installing buys deeper integration: the full skill in context, tools without discovery. Skills (recommended, works with Claude Code and Cowork):

npx skills add livevariant/livevariant

Claude Code plugin (this repository is the marketplace):

claude
/plugin marketplace add livevariant/livevariant
/plugin install livevariant@livevariant

Codex plugin:

codex plugin marketplace add livevariant/livevariant
codex plugin add livevariant/livevariant

Any other agent: MCP hosted at https://livevariant.com/mcp, stdio via npx -y @livevariant/mcp, or plain HTTP at POST /api/v1/<tool> (interactive docs at /docs, spec at /openapi.json). No API keys: a test's config and its stats secret carry all the authority there is.

Or do it yourself

The builder composes a test in the browser, no code and no account. Or write the URL by hand:

https://livevariant.link/s?v=https://cdn.you.com/hero-a.jpg&v=https://cdn.you.com/hero-b.jpg&id={{recipient_id}}

Replace your email's image URL with that, and the integration is done. Every recipient sticks to their variant across opens, traffic shifts toward the winner while the campaign runs, and clicks (/c) plus a thank-you-page pixel (/px) close the loop. Add &kh=<your-stats-key> (a stable value: generate one under Settings, or take the builder's) to make results readable with your stats secret.

Multiple elements? Slots test the combination, not isolated pieces (wrapped here for reading; variant values are full URLs):

https://livevariant.link/s?s=hero&v=https://cdn.you.com/hero-a.jpg
                          &v=https://cdn.you.com/hero-b.jpg
                          &s=product&v=https://cdn.you.com/shot-1.jpg
                          &v=https://cdn.you.com/shot-2.jpg
                          &id={{recipient_id}}&slot=hero

One link per element (&slot=), one sticky combination per recipient, and the model learns that hero A only wins with product shot 2, which two separate tests can never see.

For audience segments in email, use what survives mail proxies: campaign tags (ctx=source:utm_source) or your ESP's merge fields (&c_country={{country}}).

Test your website, too

Landing pages are tests as well: images and content served directly on the page. Developers and LLM coding agents wire one up by installing the SDK; the config is readable on purpose, and this is the whole test:

npm i @livevariant/sdk

import { createTest } from "@livevariant/sdk";

const test = await createTest(
  {
    slots: {
      headline: [
        "The daily cup, perfected",
        "Mornings, upgraded",
        "Coffee worth waking for"
      ],
      cta: ["Shop the roast", "Start your ritual", "See the blends"]
    },
    ctx: {
      dims: [
        { key: "country", from: "country" },
        { key: "device", from: "device" }
      ]
    }
  },
  { serverUrl: "https://livevariant.link" }
);

headline.textContent = test.slots.headline.text;
cta.textContent = test.slots.cta.text;
// conversions auto-tracked from your existing GA events

Two slots, nine combinations, a different winner per country and device, and the test is scoped to your domain automatically. If the server is unreachable, visitors get your control and nothing breaks. The headline on livevariant.com runs exactly this way; the page shows its own snippet.

Read your results

Building a test (through the builder or build_test) shows the stats secret exactly once; only its hash travels in the config, so nobody can recover it later, including us. Keep it.

• The manage URL carries the secret in its #fragment (which never reaches server logs): open it for live per-combination and per-slot numbers.
• Agents call get_stats with the same secret and get win probabilities plus an honest stop/continue call, instead of eyeballed conversion rates.
• A test built without a stats key still serves and learns, but its results are unreadable forever: no secret can match a hash that is not there.

Own your tests (optional)

No account is ever required, but on livevariant.com you can create one (email link or Google) and it buys three things:

• Claiming. Open any manage link while signed in and click "Add to my account": the stats key behind it is claimed to you, nobody else can claim it, and every test built from it, past and future, appears under My tests on any browser. Your stats secret keeps working; a per-key lock can additionally require sign-in if it ever leaks.
• Verified domains. Prove a domain with a DNS TXT record or a well-known file and redirects to it skip the "Redirecting you to…" confirmation screen that unverified destinations show to visitors.
• SDK auto-registration. Create a publishable pk_ key, pass it to the SDK from a verified domain, and inline tests register themselves under My tests, readable without any secret in the loop.

Why not a normal A/B test?

The classic email flow sends A to 10%, B to 10%, waits a few hours, and blasts the "winner" to the rest. Decided once, on early openers, one element at a time, one answer for everyone, and it ends.

LiveVariant keeps everyone in the test forever: allocation adapts on every serve, per segment, across combinations, and priors (yours or your LLM's) give it a head start that real data can always override. Keep your ESP's subject-line test (subjects render before anything loads); everything after the open is ours. The mechanics are the published literature (Thompson 1933; Chapelle & Li 2011; Li et al. 2010; Hill et al., KDD 2017; Shivaswamy & Joachims 2012), implemented small enough to audit.

That trade is not free, and it is worth stating plainly. Adaptive allocation buys lower regret and pays for it in measurement precision. Starving a losing variant is the point, but it also means that variant's reported rate is biased low, by about 11% of its own value in our simulations, because an arm that has an unlucky start gets sampled less and few observations arrive to correct it. And any rule you poll continuously loses the guarantee it would have at a single look. So LiveVariant optimizes earnings-while-learning rather than the precision of the number you read afterwards. For a campaign whose goal is conversions that is the right way round; if you need a defensible measurement of the gap rather than the most conversions, a fixed 50/50 split and a single pre-planned look is the better instrument.

Deploy your own

This product is designed to be self-deployed. Our server can be used for testing, but its state can be destroyed at any time; a managed hosted version is in the works. Your deployment runs the same AGPL code with none of those caveats: