Ai Model Experiments vs Pqs MCP Server
A side-by-side look at the Ai Model Experiments and Pqs MCP Server MCP servers: transport, auth, tools, quality signals, and the config to install either one in Claude, Cursor, Windsurf or VS Code.
Verdict: choose Ai Model Experiments for conversational ai work as a local process; choose Pqs MCP Server for conversational ai as a local process. You can also run both side by side in one mcpServers block.
Which MCP server should you choose?
Choose Ai Model Experiments when
- You need dedicated capabilities in the Conversational AI domain.
- You prefer a local stdio subprocess transport.
- You want tools like
experiment_models, experiment_estimate, experiment_create.
Choose Pqs MCP Server when
- You need dedicated capabilities in the Conversational AI domain.
- You prefer a local stdio subprocess transport.
- You have the keys it needs:
PQS_BASE. - You want tools like
score_prompt, optimize_prompt.
Feature & specification comparison
Rows where the two servers differ are marked.
| Spec | ||
|---|---|---|
| Category | Conversational AI | Conversational AI |
| Quality signal | 61/100 · Good | 56/100 · Good |
| Install command | Confirmed | Confirmed |
| Transport | Local subprocess (stdio) | Local subprocess (stdio) |
| Auth | Not declared | Not declared |
| Pricing | Not declared | Not declared |
| Required env vars | None listed | PQS_BASE |
| Verified owner | Community listing | Community listing |
| GitHub stars | 0 | 2 |
| Directory activity | 1 views 0 installs 0 upvotes | 2 views 0 installs 0 upvotes |
| Tools | 8 | 2 |
| Full listing | View Ai Model Experiments | View Pqs MCP Server |
Tools & capabilities
Ai Model Experiments tools 8
experiment_modelsList AI models available for experiments (about 300 across Anthropic, OpenAI, Google, Meta, Mistral, DeepSeek, Qwen and more), with context window and OUR per-token prices (provider cost × 1.5 — what experiments actually bill). Filter by name/vendor search, minimum context, or max price. Use the returned model ids in experiment_create. Example: experiment_models({ search: "claude", min_context: 100000 })experiment_estimateDry-run cost estimate for an experiment BEFORE creating it — cell count and estimated billed cost range (at our 1.5× pricing) for prompts × models × reps. Free to call, no side effects, does not need credit balance. Same spec shape as experiment_create. Example: experiment_estimate({ prompts: ["Summarize: ..."], models: ["anthropic/claude-sonnet-4.5", "openai/gpt-5"], reps: 2 })experiment_createModel Lab: run one prompt across many AI models at once and compare their outputs, cost, and latency side by side. Create and start an experiment: run each prompt against each model (× reps), collecting output, tokens, latency, and billed cost per cell. PREPAID: requires Pipeworx credit balance ≥ max_spend_usd (top up via experiment_topup); bills actual provider cost × 1.5 with a $0.10 minimum per experiment. ASYNC: returns experiment_id immediately — execution starts within ~1 minute; poll experiment_status until complete, then call experiment_results. Do NOT wait synchronously. Set summary:false to skip the AI-written model-comparison summary. Example: experiment_create({ name: "tone test", prompts: ["Rewrite formally: ..."], models: ["anthropic/claude-haiku-4.5", "openai/gpt-5-mini"], reps: 2, max_spend_usd: 2 })experiment_statusProgress of an experiment: cell counts by state (pending/running/ok/error/skipped), spend so far vs cap, and whether it is complete. Poll this after experiment_create (every few seconds). Example: experiment_status({ experiment_id: "..." })experiment_resultsResults of an experiment: per-model aggregates (mean latency, tokens, total billed cost, error rate), per-cell outputs, and the AI-written summary comparing how the models differed (if enabled). Use include_outputs:false for aggregates only. Example: experiment_results({ experiment_id: "..." })experiment_listList your experiments, newest first, with status and spend. Example: experiment_list({ limit: 10 })experiment_cancelCancel a running experiment: pending cells are skipped (not billed); in-flight cells finish and bill. Example: experiment_cancel({ experiment_id: "..." })experiment_topupHow to add prepaid credits for experiments (and your current balance). Payment is x402 — USDC on Base, paid in-band by any wallet-equipped agent: POST https://gateway.pipeworx.io/credits/topup?amount_usd=10 responds HTTP 402 with payment requirements; retry with PAYMENT-SIGNATURE to settle and the credits land instantly. Example: experiment_topup({})
Pqs MCP Server tools 2
score_promptScore a prompt's quality across 8 dimensions BEFORE sending it to an expensive model. Returns a 0-80 score, an A-F grade, the per-dimension breakdown (clarity, specificity, context, constraints, output_format, role_definition, examples, cot_structure), and the weakest dimension. USE WHEN: - The user is workshopping a prompt and asks "is this good?" / "will this work?" / "should I add more detail?" - The user is about to send a long or expensive prompt to GPT-4, Claude Opus, or any frontier model, especially in a batch or automation context where rework is costly. - The user mentions iterating on a prompt that produced poor output and wants to diagnose what's missing. - The user pastes a prompt and asks for feedback on it. DO NOT USE WHEN: - The user is asking you to write a prompt for them (write it yourself first, then optionally call score_prompt to verify). - The prompt is conversational chat (this scores task-shaped prompts). COST: Free, no API key required. Rate-limited per IP: 5/min, 10/day, 100/month. If the user exceeds the limit, the response will include a structured upgrade path with subscribe and account URLs. LATENCY: ~2 seconds.optimize_promptRewrite a prompt to score higher on the PQS rubric, AND show before/after output comparisons so the user can see the impact. Returns the optimized prompt, the original PQS score, the optimized PQS score, and side-by-side sample outputs from a frontier model using both versions. USE WHEN: - The user got a low score from score_prompt and asks how to improve. - The user explicitly asks to "improve" / "rewrite" / "fix" / "optimize" a prompt they pasted. - The user is dissatisfied with output quality from a previous prompt and asks how to get better results. - score_prompt returned a suggestion to invoke this tool. DO NOT USE WHEN: - The user just asked for a score (use score_prompt only — don't double up). - The user wants you to write a new prompt from scratch (write it directly). REQUIRES: A PQS API key from a Pro subscription ($19.99/month, 1,000 calls/mo, includes batch + A/B comparison). If the user has not provided one, the tool returns a clear subscription URL — pass that response to the user verbatim. Do not invent or guess API keys. There is no free trial of this tool; the user must subscribe before the first call. COST: Counted against your Pro subscription's monthly call quota. LATENCY: ~6-8 seconds.
Ready-to-paste client configs
Paste either block (or both) into your client config, e.g. claude_desktop_config.json or ~/.cursor/mcp.json. We only show a config when the install command is confirmed.
Ai Model Experiments
{
"mcpServers": {
"ai-model-experiments": {
"command": "npx",
"args": [
"-y",
"@pipeworx/mcp-ai-model-experiments"
]
}
}
}
Pqs MCP Server
{
"mcpServers": {
"pqs-mcp-server": {
"command": "npx",
"args": [
"-y",
"pqs-mcp-server"
],
"env": {
"PQS_BASE": "YOUR_PQS_BASE_HERE"
}
}
}
}
Frequently asked questions
Ai Model Experiments is categorized under Conversational AI and uses a local stdio subprocess. Pqs MCP Server belongs to Conversational AI and uses a local stdio subprocess. Pick Ai Model Experiments for conversational ai work and Pqs MCP Server for conversational ai.
Copy the mcpServers block from the config section on this page into your client's config (for example claude_desktop_config.json or ~/.cursor/mcp.json), then fully restart the client.
Ai Model Experiments has no declared pricing — check the project's own site. Pqs MCP Server has no declared pricing — check the project's own site. If a server needs environment variables such as API keys, define them under the env key in your client's MCP config.
Yes. MCP clients support multiple servers: add ai-model-experiments and pqs-mcp-server as separate keys in the same mcpServers object, and your assistant will call tools from either one.
On AllMCPs, Ai Model Experiments has a quality signal of 61/100 (Good) with 1 views, 0 installs and 0 GitHub stars. Pqs MCP Server has 56/100 (Good) with 2 views, 0 installs and 2 GitHub stars.