haiku.rag

Agentic RAG that answers questions about your own documents with citations to page numbers and section headings. Runs locally on an embedded database, no server required.

Built on LanceDB, Pydantic AI, and Docling. Full documentation at ggozad.github.io/haiku.rag.

Features

Hybrid search — Vector + full-text with Reciprocal Rank Fusion
Multimodal & cross-modal search — Multimodal embedders (vLLM, VoyageAI, Cohere) put picture vectors in the same space as text; supports text-as-query → figure hits and image-as-query
Question answering — RAG capability with citations (page numbers, section headings)
Vision QA — Vision-capable models receive figure bytes alongside chunk text; attach your own images to questions in ask, analyze and the chat TUI
Reranking — local cross-encoders, Cohere, Zero Entropy, or vLLM
Analysis capability — Complex analytical tasks via sandboxed Python code execution (aggregation, computation, multi-document analysis)
Evidence compaction — Optional capability that replaces earlier questions' search results on the request with the evidence they cited, so long conversations stop resending everything they retrieved
Citation policy — Optional capability that requires every answer to declare what grounds it, including declaring that nothing does
Conversational RAG — Chat TUI and web application for multi-turn conversations with session memory
Document structure — Stores full DoclingDocument, enabling structure-aware context expansion
Multiple providers — Embeddings: Ollama, OpenAI, VoyageAI, Cohere, LM Studio, vLLM (multimodal via multimodal: true on vLLM/VoyageAI/Cohere). QA: any model supported by Pydantic AI
Multi-database search — Search, ask, analyze, or chat across named databases with source attribution on results and citations
Local-first — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
CLI & Python API — Full functionality from command line or code
MCP server — Expose as tools for AI assistants (Claude Desktop, etc.)
Visual grounding — View chunks highlighted on original page images
Production ingester — Long-lived haiku-ingester service with persistent SQLite queue, async worker pool with retries and a dead-letter queue, FS / HTTP / S3 / WebDAV source adapters, FastAPI control plane, and a browser dashboard for operators. See docs/ingester.md.
Tags — Name database states with haiku-rag tag and roll back to them
Inspector — TUI for browsing documents, chunks, and search results

Installation

Python 3.12 or newer required

Full Package (Recommended)

Terminal

pip install haiku.rag

Includes all features: document processing, all embedding providers, and rerankers.

Using uv? uv pip install haiku.rag

Slim Package (Minimal Dependencies)

Terminal

pip install haiku.rag-slim

Install only the extras you need. See the Installation documentation for available options.

Quick Start

Note: Requires an embedding provider (Ollama, OpenAI, etc.). See the Tutorial for setup instructions.

bash

# Index a PDF
haiku-rag add-src paper.pdf

# Search
haiku-rag search "attention mechanism"

# Ask questions with citations
haiku-rag ask "What datasets were used for evaluation?"

# Ask about an image (vision-capable model)
haiku-rag ask "Does this figure match the spec in the design doc?" --image figure.png

# Analyze — complex analytical tasks via code execution
haiku-rag analyze "How many documents mention transformers?"

# Interactive chat — multi-turn conversations with memory
haiku-rag chat

# Continuously ingest from configured sources (FS, HTTP, S3, WebDAV)
haiku-ingester serve

See Configuration for customization options.

Python API

server.ts

from haiku.rag.client import HaikuRAG

async with HaikuRAG("knowledge.lancedb", create=True) as rag:
    # Index documents
    await rag.create_document_from_source("paper.pdf")
    await rag.create_document_from_source("https://arxiv.org/pdf/1706.03762")

    # Search — returns chunks with provenance
    results = await rag.search("self-attention")
    for result in results:
        print(f"{result.score:.2f} | p.{result.page_numbers} | {result.content[:100]}")

    # QA with citations
    answer, citations = await rag.ask("What is the complexity of self-attention?")
    print(answer)
    for cite in citations:
        print(f"  [{cite.chunk_id}] p.{cite.page_numbers}: {cite.content[:80]}")

For direct agent composition, see the capabilities documentation.

MCP Server

Use with AI assistants like Claude Code, Codex, and Claude Desktop:

bash

haiku-rag mcp --stdio

In Claude Code, install the plugin, which registers the server and a skill:

Terminal

claude plugin marketplace add ggozad/haiku.rag
claude plugin install haiku-rag

In Codex, install the same plugin from its marketplace:

bash

codex plugin marketplace add ggozad/haiku.rag
codex plugin add haiku-rag@haiku-rag

Add to your Claude Desktop configuration:

config.json

{
  "mcpServers": {
    "haiku-rag": {
      "command": "haiku-rag",
      "args": ["mcp", "--stdio"]
    }
  }
}

Provides search, document reading, and analysis tools directly in your AI assistant.

Examples

See the examples directory for working examples:

Docker Setup - Complete Docker deployment with continuous ingestion (haiku-ingester) and MCP server
Web Application - Full-stack conversational RAG with CopilotKit frontend

Documentation

Full documentation at: https://ggozad.github.io/haiku.rag/

Quickstart - Provider setup and first ingestion
Installation - Packages and extras
Configuration - YAML reference
CLI - Command reference
Python API - Complete API docs
Capabilities - Native Pydantic AI RAG and analysis capabilities
Tuning - Retrieval and answer-quality tuning
Ingester - Production ingester for continuous indexing from FS, HTTP, S3, and WebDAV
MCP - Model Context Protocol integration
Remote processing - Offload conversion to docling-serve
Applications - Chat TUI, web app, and inspector
Benchmarks - Performance benchmarks
Changelog - Version history

License

This project is licensed under the MIT License.

mcp-name: io.github.ggozad/haiku-rag