Vbj1808/retrieval-lens
š š - Black-box flight recorder for RAG retrieval inside MCP agents. Logs chunks, scores, sources and rankings so you can audit, replay and diff retrieval runs.
Quick Install
{
"mcpServers": {
"vbj1808-retrieval-lens": {
"command": "npx",
"args": [
"-y",
"vbj1808-retrieval-lens"
]
}
}
}Using an AI coding agent (Claude Code, Cursor, etc.)? Copy a ready-made prompt that tells it to fetch the setup instructions and install this server for you.
Documentation Overview
retrieval-lens
A black-box flight recorder for RAG retrieval inside MCP agents.
retrieval-lens is an MCP server that logs every retrieval step your RAG agent makes ā what chunks were retrieved, their scores, sources, and rankings ā so you can audit, replay, and diff retrieval runs after the fact.
The Problem
When a RAG agent gives a wrong answer, you need to know: did retrieval fail, or did generation fail? Right now there's no easy way to answer that. Your observability tool shows you the LLM call. It doesn't show you which chunks the model saw before it answered, what scores they had, or how retrieval changed between yesterday and today.
retrieval-lens fixes that. Every retrieval run is logged. Nothing is hidden.
Demo
When your RAG agent gives a wrong answer, ask retrieval-lens what it saw:
await mcp.call("retrieval_diff", {
run_id_a: "support-bot-before-embedding-refresh",
run_id_b: "support-bot-after-embedding-refresh",
match_by: "source"
});
See docs/demo-diff.png for real output from Claude Code.
MCP Tools
| Tool | What it does |
|---|---|
retrieval_observe | Log a retrieval run ā query, chunks, scores, sources, rankings |
retrieval_query | Replay what the model saw before a specific answer |
retrieval_diff | Compare two retrieval runs ā what changed, what score drifted |
retrieval_stats | Aggregate score distributions, top sources, runs over time |
Quickstart
Run retrieval-lens directly with npx:
npx retrieval-lens
Add retrieval-lens to Claude Code with one command:
claude mcp add retrieval-lens npx retrieval-lens
Then call retrieval_observe after every retrieval step in your RAG pipeline:
await mcp.call("retrieval_observe", {
run_id: crypto.randomUUID(),
query: "what is the refund policy?",
chunks: [
{ content: "Refunds are processed within 5 days...", score: 0.91, source: "policy.md", rank: 1 },
{ content: "Contact support for refund requests...", score: 0.74, source: "faq.md", rank: 2 }
],
pipeline_tag: "support-bot"
});
Adapters
LangChain
See examples/langchain-adapter.ts
LlamaIndex
See examples/llamaindex-adapter.ts
Why not LangSmith / Langfuse?
Those are full observability platforms. retrieval-lens is surgical:
- Local-first ā SQLite, zero signup, no data leaves your machine
- MCP-native ā one config line, works in any MCP client
- Retrieval-only ā focused on the layer where most RAG failures actually happen
Status
š§ Active development. Harness-first build using harness engineering principles.
- F05 ā scaffold
- F01 ā retrieval_observe
- F02 ā retrieval_query
- F03 ā retrieval_diff
- F04 ā retrieval_stats
License
MIT