Enterprise Internal Knowledge Base: Production Ready RAG + MCP

What Enterprise Internal Knowledge Base: Production-Ready RAG + MCP does

Enterprise Internal Knowledge Base: Production-Ready RAG + MCP provides a Retrieval-Augmented Generation pipeline over a public corpus of Veterans Affairs education manuals. The included dataset contains 238 documents and more than 9,000 chunks. A FastMCP server exposes the retrieval and generation flow to MCP clients, returning ranked chunks with citations.

The repository emphasizes operational checks around RAG quality. It includes a 110-question golden set, DeepEval scoring, Langfuse trace instrumentation, and a GitHub Actions workflow that blocks pull requests when defined retrieval or refusal metrics regress beyond configured thresholds. The sample implementation is intended to show how evaluation and observability fit around an MCP-accessible RAG system.

How it works

The ingestion path crawls KnowVA HTML and PDF content, applies source-specific normalization, preserves document structure during chunking, creates local 1024-dimensional embeddings with mxbai-embed-large, and stores the results in Postgres with pgvector. Chunk metadata can include headings, authority tier, access-control information, and content category. Contextual Retrieval enrichment through Anthropic's Batches API is available as an optional ingestion step.

For each query, the server searches pgvector for top-K candidates and applies either the mxbai reranker or FlashRank in fast evaluation mode. Claude Sonnet generates the answer, while Langfuse records a trace. The database also has a live GIN index for the content_tsv field, but query-time BM25 and reciprocal-rank fusion are not wired into the current search path.

Setup and configuration

The Enterprise Internal Knowledge Base: Production-Ready RAG + MCP MCP server runs locally with Docker, Python, and a Postgres/pgvector container. The documented quickstart starts the database with Docker Compose, creates a virtual environment, installs requirements, restores the supplied fixture dump, runs the MCP smoke test, and launches scripts/mcp_server.py over stdio.

Claude Desktop can start the server by pointing an MCP configuration entry at the Python script and its absolute path. The included fixture provides pre-embedded content, while the raw-corpus workflow runs crawling, metadata enrichment, preprocessing, chunking, embedding, and optional contextualization as separate, resume-safe scripts. The repository documents Postgres on port 5433.

Tools and capabilities

The FastMCP exposure includes:

• query, for searching the indexed corpus and producing an answer through the RAG path.
• document://{source_id}, a resource for retrieving a document by source identifier.
• cite_from_chunks, a prompt for working with retrieved chunks and citations.

Evaluation can run in a fast mode using FlashRank and a fixture database. The CI workflow tests the golden set and applies thresholds to top-1, top-k, keyword recall, and the idk_rate metric. The full evaluation uses the mxbai reranker and reports faithfulness, answer relevance, context precision, context recall, and context relevance.

Limitations and notes

This implementation is a reference and demonstration rather than a complete multi-tenant production service. Each MCP tool accepts a typed auth_context parameter, but it is currently unused, so tenant-level access enforcement is not implemented. Ingestion is single-threaded; a distributed worker pool would be needed for higher-throughput production processing.

The hybrid-search index exists, but BM25 plus reciprocal-rank fusion remains a future addition. Cost and latency tuning, tenant access control, and related production concerns are discussed in the linked writeup rather than fully implemented here. The sample corpus is public VA education content, so teams must supply and process their own corpus for other domains.