convai-analytics

Agentic analytics for Convai applications.

This repository lets MCP-capable agents, coding agents, and developers answer questions about Convai session telemetry using only a Convai API key. The recommended path is the published local MCP server, @convai/analytics-mcp, which plugs into Claude Desktop, Cursor, Codex-compatible MCP clients, and other stdio MCP hosts so agents can call typed analytics tools instead of writing custom scripts.

For developers who want direct programmatic access, the repo also includes TypeScript, Python, and CLI clients that use the same public Analytics API.

Ask questions like:

How many interactions did my characters have in the last 7 days?

How many unique end users used each character this month?

Show aggregate P50/P95/P99 latency over time for production readiness sign-off.

Explain why this interaction was slow and generate a waterfall chart.

Which processor, provider, model, or character is driving latency or errors?

The SDK calls Convai's hosted analytics API at https://analytics-api.convai.com/v1/analytics. The API resolves your API key server-side, scopes every query to your account, enforces plan and quota limits, and returns agent-friendly JSON that can be summarized, charted, or used in scripts.

Status: v0.2. The full v1 endpoint surface is wired: summary, timeseries, breakdown, sessions.list, sessions.get, interactions.get, metrics/catalog, regression-detection, and query. Convenience facades (latency, providers, errors, usage) work end to end on top of those primitives.

What This Is

• A local MCP server in packages/mcp that exposes the main analytics questions as typed agent tools.
• A TypeScript SDK in packages/typescript.
• A Python SDK in packages/python.
• A CLI in cli.
• Prompt recipes in recipes/prompts that tell an AI agent exactly which calls to make for common analytics questions.
• Chart recipes in recipes/charts that generate Vega-Lite specs or Plotly timelines for latency, usage, reliability, and concurrency analysis.

What This Is Not

• Not a client-side browser SDK. Your Convai API key is a bearer secret. Do not put it in browser JavaScript, mobile apps, public notebooks, screenshots, logs, or committed config files.
• Not a direct data-store client. All reads go through Convai's hosted Analytics API with server-side account scoping, plan checks, and quota enforcement.
• Not a write API. It only reads analytics data. It does not mutate characters, sessions, configs, or telemetry.
• Not a replacement for the Convai dashboard. It is the programmable and agent-friendly surface for deeper analysis, automation, and chart generation.

Repository Layout

convai-analytics/
├── docs/                       Concepts, auth, metrics reference, prompt/chart indexes
├── openapi/                    Snapshot of the analytics API contract
├── packages/
│   ├── typescript/             @convai/analytics SDK
│   ├── mcp/                    @convai/analytics-mcp local stdio server
│   └── python/                 convai-analytics SDK
├── cli/                        convai-analytics command line interface
├── recipes/
│   ├── prompts/                Natural-language prompt templates for AI agents
│   └── charts/                 Runnable chart-generation scripts
└── examples/                   Runnable end-to-end examples

Prerequisites

• Node.js 18+ for the TypeScript SDK, CLI, examples, and chart recipes.
• Python 3.10+ and uv for the Python SDK examples.
• A Convai API key for the account whose analytics you want to query.

Get your key from the same Convai account you use for the rest of the Convai API, then export it in your shell:

export CONVAI_API_KEY="ck_live_your_key_here"

The SDK reads CONVAI_API_KEY automatically. You normally do not need to set a base URL; production is the default.

Recommended: MCP From A Fresh Client

For agent workflows, start here:

export CONVAI_API_KEY="ck_live_your_key_here"
npx -y @convai/analytics-mcp@latest

The server is also listed in the Official MCP Registry as io.github.Conv-AI/convai-analytics-mcp. If your MCP client supports registry discovery, use that registry entry; otherwise configure the npx command above. Once connected, ask normal analytics questions:

After adding or changing MCP config, restart Claude Desktop, Claude Code, Cursor, Codex, or your MCP host so it reloads the server command and CONVAI_API_KEY.

Show aggregate P50/P95/P99 latency for the last 30 days and generate a chart.

Which component is driving p95 latency, and which sessions should I inspect?

Show usage trends, unique end users, active-session concurrency, and provider/model latency charts.

See packages/mcp/README.md for client setup snippets.

Developer SDK Path From A Fresh Clone

git clone https://github.com/Conv-AI/convai-analytics.git
cd convai-analytics
export CONVAI_API_KEY="ck_live_your_key_here"

# Install TypeScript SDK + CLI dependencies.
make ts-install

# Optional but useful: verify the SDK and CLI compile.
make ts-build
make ts-test

Run the smallest working example:

packages/typescript/node_modules/.bin/tsx \
  --tsconfig tsconfig.recipes.json \
  examples/node/account-summary.ts last_7d

Expected result: a short account summary with sessions, end users, interactions, errors, and latency percentiles for your own account.

Use With A Coding Agent

If your agent does not support MCP, open this repo in Codex, Claude Code, Cursor, or a similar coding agent. Give the agent this instruction:

Use this repository and my CONVAI_API_KEY environment variable to answer analytics questions.
First read README.md, docs/concepts.md, docs/metrics-reference.md, and recipes/prompts/README.md.
Prefer MCP when available; otherwise use the SDK or recipes. Do not ask for direct data-store access.
Keep the API key private and do not print it.

Then ask normal product questions:

How many interactions and unique end users did I have in the last 30 days?
Break it down by character and generate a usage trend chart.

Show aggregate P50/P95/P99 voice.user_to_bot_latency for the last 7 days.
Add a 3000 ms p95 threshold line and summarize the worst buckets.

List recent slow sessions, choose one with traceable interaction spans,
explain the bottleneck, and generate a waterfall chart.

The agent should use recipes/prompts for the call sequence and recipes/charts for chart generation.

MCP Server

The fastest path for MCP-capable agents is @convai/analytics-mcp. It is a published local stdio MCP server that wraps only the public TypeScript SDK. It reads CONVAI_API_KEY, optionally reads CONVAI_ANALYTICS_BASE_URL, and never accepts account overrides, service credentials, database URLs, or other internal access paths.

Run it directly:

export CONVAI_API_KEY="ck_live_your_key_here"
npx -y @convai/analytics-mcp@latest

Official MCP Registry name: io.github.Conv-AI/convai-analytics-mcp

Claude Desktop example:

{
  "mcpServers": {
    "convai-analytics": {
      "command": "npx",
      "args": ["-y", "@convai/analytics-mcp@latest"],
      "env": {
        "CONVAI_API_KEY": "ck_live_your_key_here"
      }
    }
  }
}

After saving this config, quit and reopen Claude Desktop so it starts the new MCP server. Claude Code, Cursor, Codex-compatible clients, and other stdio MCP hosts can use the same command/env shape; restart the client or start a new session after adding or changing MCP config. Once connected, ask questions like:

Show aggregate P50/P95/P99 latency for the last 30 days and generate a chart.

Which component is driving p95 latency, and which sessions should I inspect?

Show usage trends, unique end users, active-session concurrency, and provider/model latency charts.

The MCP server returns structured JSON for data tools and Vega-Lite JSON specs for chart tools. It does not write files; your MCP client can decide whether to render, save, or summarize the returned artifacts.

Local development:

make mcp-install
make mcp-lint
make mcp-test
make mcp-build
make mcp-smoke

See packages/mcp/README.md for the full tool list, prompt list, resources, and live smoke command. See docs/publishing.md and docs/mcp-distribution.md for release, registry, and marketplace publishing notes.

TypeScript SDK

When using the repository directly:

make ts-install
packages/typescript/node_modules/.bin/tsx --tsconfig tsconfig.recipes.json examples/node/account-summary.ts last_24h

When using the package from your own Node project:

npm install @convai/analytics

import { ConvaiAnalytics } from "@convai/analytics";

const client = new ConvaiAnalytics({
  apiKey: process.env.CONVAI_API_KEY,
});

const summary = await client.summary({ range: "last_7d" });
console.log({
  sessions: summary.sessions,
  interactions: summary.interactions,
  uniqueEndUsers: summary.uniqueEndUsers,
  p95EndToEndMs: summary.p95EndToEndMs,
});

Find sessions to inspect:

const sessions = await client.sessions.list({
  range: "last_7d",
  sort: "slowest",
  limit: 10,
});

console.table(
  sessions.sessions.map((s) => ({
    sessionId: s.sessionId,
    interactions: s.interactionCount,
    p95EndToEndMs: s.p95EndToEndMs,
    startTime: s.startTime,
  })),
);