DeepRepo β Local RAG Engine for Codebases
A production-grade Python library for performing RAG (Retrieval Augmented Generation) on local codebases. No heavy frameworks, no external vector DBs, no cloud required.
What It Does
DeepRepo ingests a codebase and builds three things simultaneously:
| Layer | What it stores | Used for |
|---|
| Code Knowledge Graph | Classes, functions, imports, call edges (SQLite) | Symbol lookup, blast-radius analysis |
| Embeddings + FTS index | Semantic vectors + full-text search | Relevant code retrieval |
| Hierarchical Wiki | Plain-English .md files per module | AI explanations, chat context |
A smart query router classifies every question and picks the cheapest context strategy, reducing LLM token usage by 5β50x compared to naive RAG.
Features
- Zero dependencies on heavy frameworks β pure Python, SQLite-backed
- Multiple AI providers β Ollama (free/local), OpenAI, Anthropic, Gemini, HuggingFace
- CLI-first β
deeprepo ingest . / deeprepo serve / deeprepo query "β¦"
- Wiki viewer β browsable, searchable HTML wiki with in-page chat (
deeprepo serve)
- 7 focused MCP tools β drop DeepRepo into Cursor / Claude Desktop as an MCP server
- Branch isolation β per-branch SQLite databases with copy-on-write from base branches
- 3-tier retrieval β Embeddings β FTS β Graph fallback for resilient search
- Incremental ingestion β unchanged files are skipped; only deltas re-processed
Quick Start
1. Install
cd deeprepo_core
pip install -e .
For MCP server support:
2. Install Ollama (free, local β recommended)
# macOS
brew install ollama
ollama serve # keep this running
ollama pull nomic-embed-text # embedding model
ollama pull llama3.1:8b # LLM
3. Ingest your codebase
cd /path/to/your/project
deeprepo ingest .
4. Browse the wiki
deeprepo serve # opens http://localhost:8080
5. Ask questions
deeprepo query "how does authentication work?"
deeprepo query "what breaks if I change auth.py?"
CLI Reference
deeprepo <command> [options]
| Command | What it does |
|---|
deeprepo init | Detect provider setup, print the ingest command |
deeprepo ingest [PATH] | Scan repo β build graph + wiki + embeddings |
deeprepo wiki [PATH] | Regenerate wiki pages only (skip re-indexing) |
deeprepo serve | Launch wiki viewer + in-page chat at port 8080 |
deeprepo query "QUESTION" | Ask a question, get an AI answer |
deeprepo status | Show branch isolation & cache freshness |
Common flags (all commands)
--llm ollama|openai|anthropic|gemini|huggingface # LLM provider
--embed ollama|openai|huggingface # embedding provider (default: same as --llm)
--branch-isolation # enable per-branch databases
--base-branch main # seed feature-branch cache from main
--wiki-dir .deeprepo/wiki # override wiki output directory
ingest flags
--chunk-size N # chars per text chunk (default: 1000)
--overlap N # overlap between chunks (default: 100)
--workers N # wiki parallel workers (default: 3)
--no-wiki # skip wiki generation
serve flags
--port N # HTTP port (default: 8080)
Examples
# Ollama (free, fully local)
deeprepo ingest .
# OpenAI embeddings + Anthropic LLM
deeprepo ingest . --embed openai --llm anthropic
# Branch isolation for a feature branch
deeprepo ingest . --branch-isolation --base-branch main
# Serve wiki with chat on a custom port
deeprepo serve --llm openai --port 9000
# Query with specific top-k results
deeprepo query "where is AuthService defined?" --top-k 3
Python API
from deeprepo import DeepRepoClient
# Single provider (backward-compatible shorthand)
client = DeepRepoClient(provider_name="ollama")
# Split providers β Anthropic LLM + OpenAI embeddings
client = DeepRepoClient(
embedding_provider_name="openai",
llm_provider_name="anthropic",
)
# Branch isolation (team workflow)
client = DeepRepoClient(
provider_name="ollama",
branch_isolation=True,
base_branches=["main"],
)
# Ingest (incremental β unchanged files are skipped)
result = client.ingest("/path/to/your/code")
print(f"Files: {result['files_scanned']}, Wiki pages: {result['wiki_generated']}")
# Query β smart routing selects the cheapest context strategy
response = client.query("How does authentication work?")
print(response['answer'])
print(f"Intent: {response['intent']}, Strategy: {response['strategy']}")
print(f"Sources: {response['sources']}") # list of file paths
# Browse the generated wiki
print(f"Wiki at: {client.get_wiki_dir()}")
query() return shape
{
"answer": str, # LLM-generated answer
"sources": list[str], # file paths used as context
"intent": str, # navigate | impact | explain | debug | review | general
"strategy": str, # e.g. symbol_lookup, blast_radius, wiki_plus_skeleton, β¦
"retrieval": str, # embeddings | fts | graph
"token_estimate": int, # estimated tokens consumed
"history": list[dict], # conversation history (last N exchanges)
}
Supported AI Providers
| Provider | Cost | Setup | Best For |
|---|
| Ollama | FREE, unlimited | Install app + ollama pull | Local dev, privacy, offline |
| OpenAI | Paid | OPENAI_API_KEY | Production, best quality |
| Anthropic | Paid | ANTHROPIC_API_KEY | Production, excellent reasoning |
| Gemini | Free tier | GEMINI_API_KEY | Experimentation |
| HuggingFace | Free tier | HUGGINGFACE_API_KEY | Cloud embeddings, no GPU needed |
Note: Anthropic has no embeddings API. Pair it with another provider:
client = DeepRepoClient(embedding_provider_name="openai", llm_provider_name="anthropic")
Architecture
deeprepo_core/src/deeprepo/
βββ client.py # Main facade β branch isolation, freshness, provider wiring
βββ graph.py # SQLite store: graph nodes/edges, embeddings, wiki index, state
βββ graph_builder.py # Tree-sitter AST parser β code knowledge graph
βββ wiki.py # Hierarchical wiki engine β bottom-up LLM synthesis
βββ router.py # Intent classifier + 6 context strategy selectors
βββ ingestion.py # File scanner, chunker, language detection
βββ interfaces.py # Abstract base classes (EmbeddingProvider, LLMProvider)
βββ registry.py # @register_embedding / @register_llm decorator system
βββ ui.py # Wiki viewer (HTTP server + mermaid renderer + chat)
βββ mcp/
β βββ server.py # 7 MCP tools for AI assistants (Cursor, Claude Desktop)
βββ providers/
βββ ollama_v.py
βββ openai_v.py
βββ anthropic_v.py
βββ gemini_v.py
βββ huggingface_v.py
.deeprepo/ # Generated (gitignore this)
βββ default.db # SQLite: graph + embeddings + wiki index + state
βββ <branch>.db # Per-branch database when branch_isolation=True
βββ wiki/ # Browsable .md wiki files
βββ overview.md # Whole-repo narrative overview
βββ *.md # One page per module
Storage
Everything lives in a single SQLite file per branch β no Redis, no Postgres, no Chroma.
| Table | Contents |
|---|
nodes | Files, classes, functions with metadata |
edges | Import / call relationships between nodes |
embeddings | Float vectors for semantic search |
wiki_pages | Generated wiki markdown (key β content) |
wiki_fts | Full-text search index over wiki |
state | Per-file SHA-256 hashes for incremental updates |
Design Patterns
- Facade β
DeepRepoClient is the single entry point; internals are hidden
- Strategy β
LLMProvider / EmbeddingProvider abstract interfaces; providers are swappable
- Registry β
@register_llm("ollama") decorator auto-registers providers at import time
- Bottom-up synthesis β wiki pages generated leaves-first; parent pages consume child summaries
- 3-tier fallback β Embeddings β FTS β Graph; queries work even when embeddings are cold
- Copy-on-write branching β feature branches start from base-branch cache, then delta-update
MCP Server (AI Assistant Integration)
Connect DeepRepo as an MCP server so Cursor, Claude Desktop, or any MCP-compatible AI assistant can call it directly β without ever reading raw files.
Setup
pip install deeprepo[mcp]
Cursor β create ~/.cursor/mcp.json:
{
"mcpServers": {
"deeprepo": {
"command": "python",
"args": ["-m", "deeprepo.mcp.server"],
"env": {
"LLM_PROVIDER": "ollama"
}
}
}
}
Claude Desktop β add to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"deeprepo": {
"command": "deeprepo-mcp",
"env": {
"EMBEDDING_PROVIDER": "openai",
"LLM_PROVIDER": "anthropic",
"OPENAI_API_KEY": "sk-...",
"ANTHROPIC_API_KEY": "sk-ant-..."
}
}
}
}
Available MCP Tools (7 tools)