Ai Model Experiments vs Relay
A side-by-side look at the Ai Model Experiments and Relay MCP servers: transport, auth, tools, quality signals, and the config to install either one in Claude, Cursor, Windsurf or VS Code.
Verdict: choose Ai Model Experiments for conversational ai work as a local process; choose Relay for conversational ai as a local process. You can also run both side by side in one mcpServers block.
Which MCP server should you choose?
Choose Ai Model Experiments when
- You need dedicated capabilities in the Conversational AI domain.
- You prefer a local stdio subprocess transport.
- You want tools like
experiment_models, experiment_estimate, experiment_create.
Choose Relay when
- You need dedicated capabilities in the Conversational AI domain.
- You prefer a local stdio subprocess transport.
- You have the keys it needs:
RELAY_API_KEY, RELAY_BASE_URL. - You want tools like
search_models, get_model, get_pricing.
Feature & specification comparison
Rows where the two servers differ are marked.
| Spec | ||
|---|---|---|
| Category | Conversational AI | Conversational AI |
| Quality signal | 61/100 · Good | 61/100 · Good |
| Install command | Confirmed | Confirmed |
| Transport | Local subprocess (stdio) | Local subprocess (stdio) |
| Auth | Not declared | Not declared |
| Pricing | Not declared | Not declared |
| Required env vars | None listed | RELAY_API_KEYRELAY_BASE_URL |
| Verified owner | Community listing | Community listing |
| GitHub stars | 0 | 0 |
| Directory activity | 1 views 0 installs 0 upvotes | 1 views 0 installs 0 upvotes |
| Tools | 8 | 14 |
| Full listing | View Ai Model Experiments | View Relay |
Tools & capabilities
Ai Model Experiments tools 8
experiment_modelsList AI models available for experiments (about 300 across Anthropic, OpenAI, Google, Meta, Mistral, DeepSeek, Qwen and more), with context window and OUR per-token prices (provider cost × 1.5 — what experiments actually bill). Filter by name/vendor search, minimum context, or max price. Use the returned model ids in experiment_create. Example: experiment_models({ search: "claude", min_context: 100000 })experiment_estimateDry-run cost estimate for an experiment BEFORE creating it — cell count and estimated billed cost range (at our 1.5× pricing) for prompts × models × reps. Free to call, no side effects, does not need credit balance. Same spec shape as experiment_create. Example: experiment_estimate({ prompts: ["Summarize: ..."], models: ["anthropic/claude-sonnet-4.5", "openai/gpt-5"], reps: 2 })experiment_createModel Lab: run one prompt across many AI models at once and compare their outputs, cost, and latency side by side. Create and start an experiment: run each prompt against each model (× reps), collecting output, tokens, latency, and billed cost per cell. PREPAID: requires Pipeworx credit balance ≥ max_spend_usd (top up via experiment_topup); bills actual provider cost × 1.5 with a $0.10 minimum per experiment. ASYNC: returns experiment_id immediately — execution starts within ~1 minute; poll experiment_status until complete, then call experiment_results. Do NOT wait synchronously. Set summary:false to skip the AI-written model-comparison summary. Example: experiment_create({ name: "tone test", prompts: ["Rewrite formally: ..."], models: ["anthropic/claude-haiku-4.5", "openai/gpt-5-mini"], reps: 2, max_spend_usd: 2 })experiment_statusProgress of an experiment: cell counts by state (pending/running/ok/error/skipped), spend so far vs cap, and whether it is complete. Poll this after experiment_create (every few seconds). Example: experiment_status({ experiment_id: "..." })experiment_resultsResults of an experiment: per-model aggregates (mean latency, tokens, total billed cost, error rate), per-cell outputs, and the AI-written summary comparing how the models differed (if enabled). Use include_outputs:false for aggregates only. Example: experiment_results({ experiment_id: "..." })experiment_listList your experiments, newest first, with status and spend. Example: experiment_list({ limit: 10 })experiment_cancelCancel a running experiment: pending cells are skipped (not billed); in-flight cells finish and bill. Example: experiment_cancel({ experiment_id: "..." })experiment_topupHow to add prepaid credits for experiments (and your current balance). Payment is x402 — USDC on Base, paid in-band by any wallet-equipped agent: POST https://gateway.pipeworx.io/credits/topup?amount_usd=10 responds HTTP 402 with payment requirements; retry with PAYMENT-SIGNATURE to settle and the credits land instantly. Example: experiment_topup({})
Relay tools 14
search_modelsLists Relay models by tag and/or text. Common tags: text-to-image, image-to-image, text-to-video, image-to-video, video-to-video, text-to-speech, speech-to-text, text-to-text, text-to-embedding. image-to-image also lists every image model that takes an input image. Returns name, tag, route and accepts (input media the model takes, e.g. ["image"]; [] for none) per model (at most 50). Then call get_model with a name for its input schema.get_modelReturns one model's route, request_schema (what run_model's input must follow), request_example, pricing and availability. Then call run_model with an input shaped like request_example.get_pricingReturns Relay's public list prices (USD), optionally for one model, plus media_storage: the per-file fees of the store_output / upload retention SKUs (relay1d, relay7d, …). Then estimate_cost for a number.estimate_costEstimates the USD cost of a call from the public price list: an estimate, not an invoice (the bill uses provider-reported usage and any custom pricing on your account). Pass only the usage fields the model's billing_type (see get_model or get_pricing) reads: per_token: input_tokens, output_tokens, cached_input_tokens; per_image: image_count, resolution_tier; per_second_video: duration_seconds, quality_mode, sound; per_second_audio: duration_seconds; per_character: character_count; per_media_token: media_input_tokens, media_output_tokens. Add store_output / retention / file_count for storage fees.run_modelRuns any Relay model by name (billed to your key). input follows get_model's request_schema (request_example runs as is). Every model whose schema takes async (image, video, audio) is submitted as a task: it waits up to 45 s, past that it returns a task_id — do not resubmit, call check_task. Chat models answer inline. Result links expire 1 h after completion unless store_output names a media_storage SKU from get_pricing. Pass an idempotency_key if you may retry the same submit. Needs a Relay credential.check_taskChecks a task run_model returned (no key needed: the task_id is the capability). Waits up to 30 s for it to finish; returns the result when completed, else its status: then call check_task again, never resubmit. Task results are kept 1 h after they finish.upload_fileHosts an image, video or audio file on Relay and returns its url: pass that url to any *_url input of run_model. Give exactly one of path (local server only, ≤ 100 MB), base64 (≤ 4 MB) or url (a public link Relay copies). retention: relay1h (default, free within a daily quota) or a media_storage SKU from get_pricing. Needs a Relay credential.list_workflowsList Relay's workflow templates: multi-step pipelines (e.g. script → voiceover) run as one call. Each entry has workflow_id, input_schema and its steps (step_id + model). Run one with run_workflow; inputs follow input_schema.run_workflowStart a workflow run (one submit) and wait up to wait_seconds (≤ 45) for it to finish. Completed → the final output; still running → run_id: call check_workflow_run with it, never resubmit (a new run bills again). Each step is billed as an ordinary request. webhook_url gets ONE signed workflow.completed / workflow.failed delivery when the run ends. Get workflow_id and the inputs shape from list_workflows. Needs a Relay credential.check_workflow_runPoll a workflow run by run_id, waiting up to wait_seconds (≤ 30) for it to end. Completed → the final output; failed / cancelled → the error; still running → its steps so far: call check_workflow_run again. No key needed: the run_id is the access token.cancel_workflow_runCancel a workflow run. Cancels between steps only: a step already running completes and bills. A run that has already ended answers WORKFLOW_RUN_NOT_CANCELLABLE. Needs a Relay credential.get_creditsRead the account's credit balance, active promo credits and consumption. Needs a dashboard JWT or a custom-tier superkey; a plain inference key gets PERMISSION_DENIED (403). Needs a Relay credential.- +2 more on the listing page
Ready-to-paste client configs
Paste either block (or both) into your client config, e.g. claude_desktop_config.json or ~/.cursor/mcp.json. We only show a config when the install command is confirmed.
Ai Model Experiments
{
"mcpServers": {
"ai-model-experiments": {
"command": "npx",
"args": [
"-y",
"@pipeworx/mcp-ai-model-experiments"
]
}
}
}
Relay
{
"mcpServers": {
"relay-4": {
"command": "npx",
"args": [
"-y",
"@relaygpu/mcp"
],
"env": {
"RELAY_API_KEY": "YOUR_RELAY_API_KEY_HERE",
"RELAY_BASE_URL": "YOUR_RELAY_BASE_URL_HERE"
}
}
}
}
Frequently asked questions
Ai Model Experiments is categorized under Conversational AI and uses a local stdio subprocess. Relay belongs to Conversational AI and uses a local stdio subprocess. Pick Ai Model Experiments for conversational ai work and Relay for conversational ai.
Copy the mcpServers block from the config section on this page into your client's config (for example claude_desktop_config.json or ~/.cursor/mcp.json), then fully restart the client.
Ai Model Experiments has no declared pricing — check the project's own site. Relay has no declared pricing — check the project's own site. If a server needs environment variables such as API keys, define them under the env key in your client's MCP config.
Yes. MCP clients support multiple servers: add ai-model-experiments and relay-4 as separate keys in the same mcpServers object, and your assistant will call tools from either one.
On AllMCPs, Ai Model Experiments has a quality signal of 61/100 (Good) with 1 views, 0 installs and 0 GitHub stars. Relay has 61/100 (Good) with 1 views, 0 installs and 0 GitHub stars.