RAGScore

Generate QA datasets & evaluate RAG systems. Privacy-first, any LLM, local or cloud.

Quick Install

Automated & IDE Setup

Copy the AI prompt to install this server into Claude Code, Cursor, or another agent — or use 1-click editor setup below.

Add to Cursor Add to VS Code

Not yet automatically verified

We haven't yet run this listing's install command through our automated sandbox check. This isn't a red flag — we're steadily working through the catalog.

Manual Client & Custom JSON ConfigExpand JSON ▾

Client Config & Setup

Choose your client or environment

Target File:~/Library/Application Support/Claude/claude_desktop_config.json

claude_desktop_config.json

{
  "mcpServers": {
    "ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ]
    }
  }
}

💡 Paste the JSON block into your client's configuration file under mcpServers, then restart the application.

Documentation Overview

Generate QA datasets & evaluate RAG systems in 2 commands

🔒 Privacy-First • ⚡ Lightning Fast • 🤖 Any LLM • 🏠 Local or Cloud • 🌍 Multilingual

English | 中文 | 日本語 | Deutsch

⚡ 2-Line RAG Evaluation

bash

# Step 1: Generate QA pairs from your docs
ragscore generate docs/

# Step 2: Evaluate your RAG system
ragscore evaluate http://localhost:8000/query

That's it. Get accuracy scores and incorrect QA pairs instantly.

Code

============================================================
✅ EXCELLENT: 85/100 correct (85.0%)
Average Score: 4.20/5.0
============================================================

❌ 15 Incorrect Pairs:

  1. Q: "What is RAG?"
     Score: 2/5 - Factually incorrect

  2. Q: "How does retrieval work?"
     Score: 3/5 - Incomplete answer

🚀 Quick Start

Install

Terminal

pip install ragscore              # Core (works with Ollama)
pip install "ragscore[openai]"    # + OpenAI support
pip install "ragscore[notebook]"  # + Jupyter/Colab support
pip install "ragscore[all]"       # + All providers

Already installed? Keep up to date — new versions add features like failure diagnosis and retrieved context capture:
Terminal
pip install --upgrade ragscore

Option 1: Python API (Notebook-Friendly)

Perfect for Jupyter, Colab, and rapid iteration. Get instant visualizations.

server.ts

from ragscore import quick_test

# 1. Audit your RAG in one line
result = quick_test(
    endpoint="http://localhost:8000/query",  # Your RAG API
    docs="docs/",                            # Your documents
    n=10,                                    # Number of test questions
)

# 1b. Tailored QA — target specific audiences
result = quick_test(
    endpoint="http://localhost:8000/query",
    docs="docs/",
    audience="developers",                   # Who asks the questions?
    purpose="api-integration",               # What's the document for?
)

# 2. See the report
result.plot()

# 3. Inspect failures
bad_rows = result.df[result.df['score'] < 3]
display(bad_rows[['question', 'rag_answer', 'reason']])

Rich Object API:

result.accuracy - Accuracy score
result.df - Pandas DataFrame of all results
result.plot() - 3-panel visualization (4-panel with detailed=True)
result.corrections - List of items to fix

Option 2: CLI (Production)

Generate QA Pairs

server.ts

# Set API key (or use local Ollama - no key needed!)
export OPENAI_API_KEY="sk-..."

# Generate from any document
ragscore generate paper.pdf
ragscore generate docs/*.pdf --concurrency 10

# Tailored QA generation — target specific audiences
ragscore generate docs/ --audience developers --purpose faq
ragscore generate docs/ --audience customers --purpose "pre-sales"
ragscore generate docs/ --audience "compliance auditors" --purpose "security audit"

Evaluate Your RAG

bash

# Point to your RAG endpoint
ragscore evaluate http://localhost:8000/query

# Custom options
ragscore evaluate http://api/ask --model gpt-4o --output results.json

🔬 Detailed Multi-Metric Evaluation

Go beyond a single score. Add detailed=True to get 5 diagnostic dimensions per answer — in the same single LLM call.

python

result = quick_test(
    endpoint=my_rag,
    docs="docs/",
    n=10,
    detailed=True,  # ⭐ Enable multi-metric evaluation
)

# Inspect per-question metrics
display(result.df[[
    "question", "score", "correctness", "completeness",
    "relevance", "conciseness", "faithfulness"
]])

# Radar chart + 4-panel visualization
result.plot()

Code

==================================================
✅ PASSED: 9/10 correct (90%)
Average Score: 4.3/5.0
Threshold: 70%
──────────────────────────────────────────────────
  Correctness: 4.5/5.0
  Completeness: 4.2/5.0
  Relevance: 4.8/5.0
  Conciseness: 4.1/5.0
  Faithfulness: 4.6/5.0
==================================================

Metric	What it measures	Scale
Correctness	Semantic match to golden answer	5 = fully correct
Completeness	Covers all key points	5 = fully covered
Relevance	Addresses the question asked	5 = perfectly on-topic
Conciseness	Focused, no filler	5 = concise and precise
Faithfulness	No fabricated claims	5 = fully faithful

CLI:

bash

ragscore evaluate http://localhost:8000/query --detailed

🔍 Failure Diagnosis (`--diagnose`)

When answers fail, --diagnose tells you why — retriever miss, generator hallucination, incomplete answer, or wrong interpretation:

bash

ragscore evaluate http://localhost:8000/query --diagnose

Code

🔍 Failure Diagnosis:
  Retriever Miss: 3 (42.9%)
  Generator Hallucination: 2 (28.6%)
  Incomplete Answer: 1 (14.3%)
  Wrong Interpretation: 1 (14.3%)

Uses the support_span already generated with each QA pair to give the judge grounding context. Combine with --detailed for full diagnostics:

bash

ragscore evaluate http://localhost:8000/query --diagnose --detailed -o results.json

Category	Meaning
Retriever Miss	RAG didn't retrieve the chunk containing the evidence
Generator Hallucination	Retrieved correctly but fabricated information
Incomplete Answer	Retrieved correctly but answer is partial
Wrong Interpretation	Retrieved correctly but misunderstood the content

📓 Full demo notebook — build a mini RAG and test it with detailed metrics.

🎯 Audience & Purpose demo — generate tailored QA for developers, customers, auditors, and more.

🏠 Ollama local demo — 100% private RAG evaluation with no API keys.

🏠 100% Private with Local LLMs

bash

# Use Ollama - no API keys, no cloud, 100% private
ollama pull llama3.1
ragscore generate confidential_docs/*.pdf
ragscore evaluate http://localhost:8000/query

Perfect for: Healthcare 🏥 • Legal ⚖️ • Finance 🏦 • Research 🔬

Ollama Model Recommendations

RAGScore generates complex structured QA pairs (question + answer + rationale + support span) in JSON format. This requires models with strong instruction-following and JSON output capabilities.

Model	Size	Min RAM	QA Quality	Recommended
`llama3.1:70b`	40GB	48GB VRAM	Excellent	GPU server (A100, L40)
`qwen2.5:32b`	18GB	24GB VRAM	Excellent	GPU server (A10, L20)
`llama3.1:8b`	4.7GB	8GB VRAM	Good	Best local choice
`qwen2.5:7b`	4.4GB	8GB VRAM	Good	Good local alternative
`mistral:7b`	4.1GB	8GB VRAM	Good	Good local alternative
`llama3.2:3b`	2.0GB	4GB RAM	Fair	CPU-only / testing
`qwen2.5:1.5b`	1.0GB	2GB RAM	Poor	Not recommended

Minimum recommended: 8B+ models. Smaller models (1.5B–3B) produce lower quality support spans and may timeout on longer chunks.

Ollama Performance Guide

bash

# Recommended: 8B model with concurrency 2 for local machines
ollama pull llama3.1:8b
ragscore generate docs/ --provider ollama --model llama3.1:8b

# GPU server (A10/L20): larger model with higher concurrency
ollama pull qwen2.5:32b
ragscore generate docs/ --provider ollama --model qwen2.5:32b --concurrency 5

Expected performance (28 chunks, 5 QA pairs per chunk):

Hardware	Model	Time	Concurrency
MacBook (CPU)	llama3.2:3b	~45 min	2
MacBook (CPU)	llama3.1:8b	~25 min	2
A10 (24GB)	llama3.1:8b	~3–5 min	5
L20/L40 (48GB)	qwen2.5:32b	~3–5 min	5
OpenAI API	gpt-4o-mini	~2 min	10

RAGScore auto-reduces concurrency to 2 for local Ollama to avoid GPU/CPU contention.

🔌 Supported LLMs

Provider	Setup	Notes
Ollama	`ollama serve`	Local, free, private
OpenAI	`export OPENAI_API_KEY="sk-..."`	Best quality
Anthropic	`export ANTHROPIC_API_KEY="..."`	Long context
DashScope	`export DASHSCOPE_API_KEY="..."`	Qwen models
vLLM	`export LLM_BASE_URL="..."`	Production-grade
Any OpenAI-compatible	`export LLM_BASE_URL="..."`	Groq, Together, etc.

📊 Output Formats

Generated QA Pairs (`output/generated_qas.jsonl`)

config.json

{
  "id": "abc123",
  "question": "What is RAG?",
  "answer": "RAG (Retrieval-Augmented Generation) combines...",
  "rationale": "This is explicitly stated in the introduction...",
  "support_span": "RAG systems retrieve relevant documents...",
  "difficulty": "medium",
  "source_path": "docs/rag_intro.pdf"
}

Evaluation Results (`--output results.json`)

Read the full README →View source on GitHub →

Related MCP Servers

View all in Knowledge & Memory View all alternatives

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars

Stargazers on the source repository.

Last commit

3mo ago

Most recent push to the default branch.

Reviews

No reviews yet — be the first to share how this listing worked for you.

Frequently Asked Questions about RAGScore

Add the following block to your claude_desktop_config.json under mcpServers: "mcpServers": { "ragscore": { "command": "uvx", "args": ["ragscore"] } }

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:

Live Dynamic SVG Preview

Markdown (GitHub README)

[![AllMCPs](https://allmcps.com/api/badge/ragscore?style=directory)](https://allmcps.com/mcp/ragscore)

HTML Embed

<a href="https://allmcps.com/mcp/ragscore"><img src="https://allmcps.com/api/badge/ragscore?style=directory" alt="RAGScore on AllMCPs" /></a>

Technical Specs & Signals

CategoryKnowledge & Memory

16/16 checks healthy over the last 45d

39Quality signal: Fair · 39/100How this signal is calculated ▾

Server availabilityNot measured

Not scored for repo-hosted servers — we can't reach the running server, only its GitHub page. Hosted MCP endpoints are health-checked live.

Verified ownership10/20

Documentation & tools16/30

Adoption & activity3/15

Community engagement0/10

A guidance signal from public completeness & health data — not a user rating. New listings start lower and rise as they add docs, get verified, and grow adoption. Signals we can't observe for a listing are skipped, not counted against it.

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 1d ago via OSV.dev · ragscore (PyPI)

★ Featured

AllMCPs Server

The official MCP server for AllMCPs.com - submit and manage tools directly from your AI. The open directory for MCP servers. Connect Claude, Cursor, Windsurf, and AI agents to databases, tools, files, and APIs. Explore 10,000+ servers. AllMCPs is the premier, open directory for discovering, evaluating, and installing Model Context Protocol (MCP) servers to equip AI agents and LLMs with real-world superpowers.

Explore Server →

Own this project?

This directory is pre-filled from public sources. Claim via GitHub README, site badge, or DNS TXT to unlock edit access and the Official badge — proof is checked automatically, then reviewed by our team.

Free dofollow backlink: add your website and place the AllMCPs badge on it — no claim needed. We detect it automatically and keep it verified as long as the badge stays live.

Claim & get free dofollow

Share & Embed

Add our SVG badge (dark/light directory styles) or embeddable widget to your site.

Explore more

More in 🧠 Knowledge & Memory →Best MCP servers for Memory & Knowledge →Alternatives to RAGScore →Install in Claude Desktop Install in Cursor Install in VS Code Setup guides for all 13 MCP clients

RAGScore

Generate QA datasets & evaluate RAG systems. Privacy-first, any LLM, local or cloud.

Quick Install

Automated & IDE Setup

Copy the AI prompt to install this server into Claude Code, Cursor, or another agent — or use 1-click editor setup below.

Add to Cursor Add to VS Code

Not yet automatically verified

We haven't yet run this listing's install command through our automated sandbox check. This isn't a red flag — we're steadily working through the catalog.

Manual Client & Custom JSON ConfigExpand JSON ▾

Client Config & Setup

Choose your client or environment

Target File:~/Library/Application Support/Claude/claude_desktop_config.json

claude_desktop_config.json

{
  "mcpServers": {
    "ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ]
    }
  }
}

💡 Paste the JSON block into your client's configuration file under mcpServers, then restart the application.

Documentation Overview

Generate QA datasets & evaluate RAG systems in 2 commands

🔒 Privacy-First • ⚡ Lightning Fast • 🤖 Any LLM • 🏠 Local or Cloud • 🌍 Multilingual

English | 中文 | 日本語 | Deutsch

⚡ 2-Line RAG Evaluation

bash

# Step 1: Generate QA pairs from your docs
ragscore generate docs/

# Step 2: Evaluate your RAG system
ragscore evaluate http://localhost:8000/query

That's it. Get accuracy scores and incorrect QA pairs instantly.

Code

============================================================
✅ EXCELLENT: 85/100 correct (85.0%)
Average Score: 4.20/5.0
============================================================

❌ 15 Incorrect Pairs:

  1. Q: "What is RAG?"
     Score: 2/5 - Factually incorrect

  2. Q: "How does retrieval work?"
     Score: 3/5 - Incomplete answer

🚀 Quick Start

Install

Terminal

pip install ragscore              # Core (works with Ollama)
pip install "ragscore[openai]"    # + OpenAI support
pip install "ragscore[notebook]"  # + Jupyter/Colab support
pip install "ragscore[all]"       # + All providers

Already installed? Keep up to date — new versions add features like failure diagnosis and retrieved context capture:
Terminal
pip install --upgrade ragscore

Option 1: Python API (Notebook-Friendly)

Perfect for Jupyter, Colab, and rapid iteration. Get instant visualizations.

server.ts

from ragscore import quick_test

# 1. Audit your RAG in one line
result = quick_test(
    endpoint="http://localhost:8000/query",  # Your RAG API
    docs="docs/",                            # Your documents
    n=10,                                    # Number of test questions
)

# 1b. Tailored QA — target specific audiences
result = quick_test(
    endpoint="http://localhost:8000/query",
    docs="docs/",
    audience="developers",                   # Who asks the questions?
    purpose="api-integration",               # What's the document for?
)

# 2. See the report
result.plot()

# 3. Inspect failures
bad_rows = result.df[result.df['score'] < 3]
display(bad_rows[['question', 'rag_answer', 'reason']])

Rich Object API:

result.accuracy - Accuracy score
result.df - Pandas DataFrame of all results
result.plot() - 3-panel visualization (4-panel with detailed=True)
result.corrections - List of items to fix

Option 2: CLI (Production)

Generate QA Pairs

server.ts

# Set API key (or use local Ollama - no key needed!)
export OPENAI_API_KEY="sk-..."

# Generate from any document
ragscore generate paper.pdf
ragscore generate docs/*.pdf --concurrency 10

# Tailored QA generation — target specific audiences
ragscore generate docs/ --audience developers --purpose faq
ragscore generate docs/ --audience customers --purpose "pre-sales"
ragscore generate docs/ --audience "compliance auditors" --purpose "security audit"

Evaluate Your RAG

bash

# Point to your RAG endpoint
ragscore evaluate http://localhost:8000/query

# Custom options
ragscore evaluate http://api/ask --model gpt-4o --output results.json

🔬 Detailed Multi-Metric Evaluation

Go beyond a single score. Add detailed=True to get 5 diagnostic dimensions per answer — in the same single LLM call.

python

result = quick_test(
    endpoint=my_rag,
    docs="docs/",
    n=10,
    detailed=True,  # ⭐ Enable multi-metric evaluation
)

# Inspect per-question metrics
display(result.df[[
    "question", "score", "correctness", "completeness",
    "relevance", "conciseness", "faithfulness"
]])

# Radar chart + 4-panel visualization
result.plot()

Code

==================================================
✅ PASSED: 9/10 correct (90%)
Average Score: 4.3/5.0
Threshold: 70%
──────────────────────────────────────────────────
  Correctness: 4.5/5.0
  Completeness: 4.2/5.0
  Relevance: 4.8/5.0
  Conciseness: 4.1/5.0
  Faithfulness: 4.6/5.0
==================================================

Metric	What it measures	Scale
Correctness	Semantic match to golden answer	5 = fully correct
Completeness	Covers all key points	5 = fully covered
Relevance	Addresses the question asked	5 = perfectly on-topic
Conciseness	Focused, no filler	5 = concise and precise
Faithfulness	No fabricated claims	5 = fully faithful

CLI:

bash

ragscore evaluate http://localhost:8000/query --detailed

🔍 Failure Diagnosis (`--diagnose`)

When answers fail, --diagnose tells you why — retriever miss, generator hallucination, incomplete answer, or wrong interpretation:

bash

ragscore evaluate http://localhost:8000/query --diagnose

Code

🔍 Failure Diagnosis:
  Retriever Miss: 3 (42.9%)
  Generator Hallucination: 2 (28.6%)
  Incomplete Answer: 1 (14.3%)
  Wrong Interpretation: 1 (14.3%)

Uses the support_span already generated with each QA pair to give the judge grounding context. Combine with --detailed for full diagnostics:

bash

ragscore evaluate http://localhost:8000/query --diagnose --detailed -o results.json

Category	Meaning
Retriever Miss	RAG didn't retrieve the chunk containing the evidence
Generator Hallucination	Retrieved correctly but fabricated information
Incomplete Answer	Retrieved correctly but answer is partial
Wrong Interpretation	Retrieved correctly but misunderstood the content

📓 Full demo notebook — build a mini RAG and test it with detailed metrics.

🎯 Audience & Purpose demo — generate tailored QA for developers, customers, auditors, and more.

🏠 Ollama local demo — 100% private RAG evaluation with no API keys.

🏠 100% Private with Local LLMs

bash

# Use Ollama - no API keys, no cloud, 100% private
ollama pull llama3.1
ragscore generate confidential_docs/*.pdf
ragscore evaluate http://localhost:8000/query

Perfect for: Healthcare 🏥 • Legal ⚖️ • Finance 🏦 • Research 🔬

Ollama Model Recommendations

RAGScore generates complex structured QA pairs (question + answer + rationale + support span) in JSON format. This requires models with strong instruction-following and JSON output capabilities.

Model	Size	Min RAM	QA Quality	Recommended
`llama3.1:70b`	40GB	48GB VRAM	Excellent	GPU server (A100, L40)
`qwen2.5:32b`	18GB	24GB VRAM	Excellent	GPU server (A10, L20)
`llama3.1:8b`	4.7GB	8GB VRAM	Good	Best local choice
`qwen2.5:7b`	4.4GB	8GB VRAM	Good	Good local alternative
`mistral:7b`	4.1GB	8GB VRAM	Good	Good local alternative
`llama3.2:3b`	2.0GB	4GB RAM	Fair	CPU-only / testing
`qwen2.5:1.5b`	1.0GB	2GB RAM	Poor	Not recommended

Minimum recommended: 8B+ models. Smaller models (1.5B–3B) produce lower quality support spans and may timeout on longer chunks.

Ollama Performance Guide

bash

# Recommended: 8B model with concurrency 2 for local machines
ollama pull llama3.1:8b
ragscore generate docs/ --provider ollama --model llama3.1:8b

# GPU server (A10/L20): larger model with higher concurrency
ollama pull qwen2.5:32b
ragscore generate docs/ --provider ollama --model qwen2.5:32b --concurrency 5

Expected performance (28 chunks, 5 QA pairs per chunk):

Hardware	Model	Time	Concurrency
MacBook (CPU)	llama3.2:3b	~45 min	2
MacBook (CPU)	llama3.1:8b	~25 min	2
A10 (24GB)	llama3.1:8b	~3–5 min	5
L20/L40 (48GB)	qwen2.5:32b	~3–5 min	5
OpenAI API	gpt-4o-mini	~2 min	10

RAGScore auto-reduces concurrency to 2 for local Ollama to avoid GPU/CPU contention.

🔌 Supported LLMs

Provider	Setup	Notes
Ollama	`ollama serve`	Local, free, private
OpenAI	`export OPENAI_API_KEY="sk-..."`	Best quality
Anthropic	`export ANTHROPIC_API_KEY="..."`	Long context
DashScope	`export DASHSCOPE_API_KEY="..."`	Qwen models
vLLM	`export LLM_BASE_URL="..."`	Production-grade
Any OpenAI-compatible	`export LLM_BASE_URL="..."`	Groq, Together, etc.

📊 Output Formats

Generated QA Pairs (`output/generated_qas.jsonl`)

config.json

{
  "id": "abc123",
  "question": "What is RAG?",
  "answer": "RAG (Retrieval-Augmented Generation) combines...",
  "rationale": "This is explicitly stated in the introduction...",
  "support_span": "RAG systems retrieve relevant documents...",
  "difficulty": "medium",
  "source_path": "docs/rag_intro.pdf"
}

Evaluation Results (`--output results.json`)

Read the full README →View source on GitHub →

Related MCP Servers

View all in Knowledge & Memory View all alternatives

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars

Stargazers on the source repository.

Last commit

3mo ago

Most recent push to the default branch.

Reviews

No reviews yet — be the first to share how this listing worked for you.

Frequently Asked Questions about RAGScore

Add the following block to your claude_desktop_config.json under mcpServers: "mcpServers": { "ragscore": { "command": "uvx", "args": ["ragscore"] } }

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:

Live Dynamic SVG Preview

Markdown (GitHub README)

[![AllMCPs](https://allmcps.com/api/badge/ragscore?style=directory)](https://allmcps.com/mcp/ragscore)

HTML Embed

<a href="https://allmcps.com/mcp/ragscore"><img src="https://allmcps.com/api/badge/ragscore?style=directory" alt="RAGScore on AllMCPs" /></a>

Technical Specs & Signals

CategoryKnowledge & Memory

16/16 checks healthy over the last 45d

39Quality signal: Fair · 39/100How this signal is calculated ▾

Server availabilityNot measured

Not scored for repo-hosted servers — we can't reach the running server, only its GitHub page. Hosted MCP endpoints are health-checked live.

Verified ownership10/20

Documentation & tools16/30

Adoption & activity3/15

Community engagement0/10

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 1d ago via OSV.dev · ragscore (PyPI)

★ Featured

AllMCPs Server

Explore Server →

Own this project?

Free dofollow backlink: add your website and place the AllMCPs badge on it — no claim needed. We detect it automatically and keep it verified as long as the badge stays live.

Claim & get free dofollow

Share & Embed

Add our SVG badge (dark/light directory styles) or embeddable widget to your site.

RAGScore

Quick Install

Client Config & Setup

Documentation Overview

⚡ 2-Line RAG Evaluation

🚀 Quick Start

Install

Option 1: Python API (Notebook-Friendly)

Option 2: CLI (Production)

Generate QA Pairs

Evaluate Your RAG

🔬 Detailed Multi-Metric Evaluation

🔍 Failure Diagnosis (--diagnose)

🏠 100% Private with Local LLMs

Ollama Model Recommendations

Ollama Performance Guide

🔌 Supported LLMs

📊 Output Formats

Generated QA Pairs (output/generated_qas.jsonl)

Evaluation Results (--output results.json)

Related MCP Servers

Adoption & maintenance

Reviews

Frequently Asked Questions about RAGScore

AllMCPs Directory Badge

Technical Specs & Signals

Supply-chain signal

AllMCPs Server

Own this project?

Share & Embed

Explore more

RAGScore

Quick Install

Client Config & Setup

Documentation Overview

⚡ 2-Line RAG Evaluation

🚀 Quick Start

Install

Option 1: Python API (Notebook-Friendly)

Option 2: CLI (Production)

Generate QA Pairs

Evaluate Your RAG

🔬 Detailed Multi-Metric Evaluation

🔍 Failure Diagnosis (--diagnose)

🏠 100% Private with Local LLMs

Ollama Model Recommendations

Ollama Performance Guide

🔌 Supported LLMs

📊 Output Formats

Generated QA Pairs (output/generated_qas.jsonl)

Evaluation Results (--output results.json)

Related MCP Servers

Adoption & maintenance

Reviews

Frequently Asked Questions about RAGScore

AllMCPs Directory Badge

Technical Specs & Signals

Supply-chain signal

AllMCPs Server

Own this project?

Share & Embed

Explore more

🔍 Failure Diagnosis (`--diagnose`)

Generated QA Pairs (`output/generated_qas.jsonl`)

Evaluation Results (`--output results.json`)

🔍 Failure Diagnosis (`--diagnose`)

Generated QA Pairs (`output/generated_qas.jsonl`)

Evaluation Results (`--output results.json`)