Lilbee README

The whole local AI stack in one executable: it runs and manages the models, and searches everything you own with them.

Project site  ·  Tutorial reels  ·  PyPI  ·  Obsidian plugin  ·  REST API  ·  Chat (#lilbee)

lilbee runs and manages your models: chat, embedding, vision, and rerank, placed across every GPU you have. It puts them to work as a search engine you can talk to, over your files, notes, code, and the web, where every answer cites the exact file and line. It crawls websites into your library, launches your coding agents on local models, and hands any MCP-aware agent cited answers from everything you've indexed. The same engine backs the Obsidian community plugin, so your vault gets all of it without a terminal. Ask in plain English. No containers, no networking, nothing else to install or set up.

And it is private. Your files, the index, the embeddings, your questions, and the answers stay on your machine. lilbee sends no telemetry, needs no account, and makes no cloud call unless you configure a cloud model yourself.

It's all one program: no separate model server, vector database, or container to stand up. lilbee runs the models and keeps the index itself. Reach it as a terminal app, CLI, Model Context Protocol server, HTTP API, or Python library. Close it and it's gone, or run it as a service to keep it warm. Everything runs on your computer; it uses a cloud model only when you pick one.

Models are no different: lilbee has its own model manager and multi-GPU fleet, built on llama.cpp, so one executable does everything (browse Hugging Face, download a model, give it a role, run it on Metal / Vulkan / CUDA). You don't need Ollama or LM Studio at all: the architectures lilbee runs are behind most of the 190,000+ GGUF repos on Hugging Face, with representatives verified on real GPUs. If you already use them, point lilbee at your existing setup and keep your models.

Tutorial reel: every demo on this page, plus the extras that don't fit here, with long-form captions at lilbee.sh/tutorial.

⚠️ Beta software

lilbee is in active beta development. Every release on PyPI is a pre-release; you must use --pre (or uv's --prerelease=allow) when installing. Interfaces, command names, and on-disk formats may shift between betas. Feedback, bug reports, and issues are very welcome; that's the whole point of the beta.

Latest pre-release (always): lilbee on PyPI →

• Quick start
• Tutorial reel (long-form videos)
• Install
• Hardware requirements
• Highlights
• Why lilbee
• First start
• What you can do with it
• Agents
• TUI
• HTTP Server · REST API reference
• Supported formats
• Wiki
• Built on

Install

Two routes, and the difference matters:

• A self-contained bundle (start here): the standalone binary, or the Homebrew / AUR / Nix / Docker / Flatpak / Snap / Scoop builds that wrap it. It carries its own Python runtime, llama.cpp, the model engine, and the optional extras, so there is nothing else to install and nothing to assemble. The trade-off is one large download and a small cold-start cost the first time it self-extracts.
• Into your own Python with pip or uv (Python 3.11 to 3.14), in its own section below. This is the developer route: it installs lilbee as a library in an environment you manage, and the model engine is a separate extra from a separate index that you have to ask for. Take it if you are working on lilbee or importing it into your own code.

Have a discrete GPU? The bundled builds and the Vulkan engine already use it. There are faster vendor builds too: NVIDIA (CUDA) and AMD (ROCm).

No external services either way; lilbee downloads and runs models locally. Optional, for scanned-PDF / image OCR: Tesseract (brew install tesseract / apt install tesseract-ocr) or a GGUF vision model.

On NVIDIA hardware

The default Vulkan build works on NVIDIA cards, but there's a dedicated CUDA build that's faster on NVIDIA hardware and sidesteps the iGPU + dGPU Vulkan-loader crash on Windows.

On AMD hardware

The default Vulkan build works on AMD cards, and stays the fallback if ROCm isn't set up. There's also a dedicated ROCm build that's faster. It bundles the ROCm userspace, so you need only the amdgpu kernel driver.

Then check it runs and pick a model:

lilbee self-check    # ~90 MB download; runs an inference + an embedding; "SELF-CHECK PASSED" on success
lilbee               # launch the terminal app; pick your models from the catalog on first run

The usage guide covers the rest: TUI screens, slash commands, CLI, HTTP server, MCP, env vars, and config.toml. The settings reference lists every setting and which of those surfaces can change it.

On older CPUs (pre-AVX2)

Pre-2013 Intel or pre-Zen AMD CPUs lack AVX2, so the normal build crashes on launch. The lilbee-compat build runs on any x86-64 chip back to ~2008.

Linux runtime requirements

The Linux x86_64 wheel and binary bundle the Vulkan loader, so they start on any distro. GPU detection still reads adapters through the system loader, though: without libvulkan1 lilbee treats the machine as CPU-only. Most desktop distros (Ubuntu 22.04+, Pop!_OS, Mint) ship it; on bare Arch / Fedora / Alpine images install it once: sudo pacman -S vulkan-icd-loader (Arch / Manjaro), sudo dnf install vulkan-loader (Fedora, RHEL), or sudo apt-get install libvulkan1 (Debian, Ubuntu).

Optional extras

These only matter for a pip or uv install: add the name in brackets, e.g. pip install --pre 'lilbee[engine,crawler,litellm]' (combine multiple, and --extra-index-url still works). The standalone binary and the Homebrew / AUR / Nix / Docker / Flatpak / Snap builds already include all four, which is why they need none of this. [engine] is the one that is not really optional; lilbee works fine without the other three.

See the full guide on optional extras for configuration.

Developer install: pip and uv

For working on lilbee, or importing it as a library into an environment you manage. Everything below needs the [engine] extra and its index; the bundled builds above do not.

Hardware requirements

Standalone mode runs entirely on your machine. No cloud required. Minimum: Apple Silicon Mac, or a 64-bit Intel/AMD CPU from 2013+ (older CPUs: On older CPUs), or an ARMv8 Linux box; 8 GB RAM, 2 GB disk.

Quick start

Two recommended ways to use lilbee, depending on whether you're the one driving:

• Run lilbee for the full-screen terminal app. On first run you pick a chat model straight from the catalog, fit-checked against your hardware, then you index files, search, and chat without leaving the TUI. The Settings screen exposes every retrieval knob (search depth, distance threshold, reranker, chunking) so you can tune lilbee to your library shape.
• Connect it to your agent over MCP. Any MCP-aware coding agent calls lilbee_search / lilbee_add and gets back cited snippets it can quote. Agents can also fine-tune lilbee on the fly via lilbee_settings_set. Drop in the lilbee-mcp skill and the agent reads the full surface: every tool, every retrieval knob, and when to widen for prose vs narrow for code. See A reference for AI agents.

Retrieval defaults are sane, and every setting is tunable from env vars or config.toml, most of them from the TUI, MCP, and the HTTP API too; the settings reference has a column per surface. The CLI and HTTP API cover scripting and headless runs. See the usage guide.

Highlights

• Private by default. The models, the index, and every question and answer stay on your machine. No telemetry, no account, no cloud calls unless you add a cloud model yourself.
• Answers cite the source line. Click a citation, jump to the file at the exact line; when the answer isn't in your library, lilbee says so instead of inventing one.
• It works, and the demos prove it. Every GIF and reel here is recorded live on real hardware, nothing staged, backed by 100% test coverage, full typing, and CI on macOS, Linux, and Windows.
• One command to running. Install, run lilbee, pick a model that fits your machine from the catalog, and you're chatting.
• Reads almost anything: 90+ formats and 150+ languages across documents, scanned pages, spreadsheets, ebooks, web pages, and source code.
• Chunks that stand on their own. Prose and code are split differently so each piece keeps its meaning, which is where most of the retrieval quality lives.
• A real search engine on top, ranking every result by how well it answers you, with 50+ tunable knobs and sane defaults.
• It brings and runs the models itself, on Metal, Vulkan, or CUDA, with no server to point at and no cloud account. Browse Hugging Face, pull a model, give it a role (chat, embedding, vision, rerank).
• A model too big for one card runs across all of them, sized with gguf-parser and tensor-split automatically, or pinned by hand. Run a model bigger than one card.
• Already on Ollama or LM Studio? Keep them. lilbee's own manager covers the same architectures they run, and their models also show up in the same pickers.
• One install, many surfaces: TUI, CLI, MCP server, REST API, and Python library, so your coding agent answers from your real files, with citations.
• Everything in one file, nothing to operate. The binary bundles the whole stack (search engine, crawler, MCP + HTTP servers, TUI, Python, llama.cpp) in ~360-530 MB, or ~0.7-1.3 GB with CUDA; it loads on demand and nothing stays running.
• Per-project libraries. One library for everything, or one per project.

Why lilbee

A small local model is fun, but limited on its own. Give it properly processed documents and a search engine over them, and it becomes genuinely powerful. Without those, it never gets past novelty.

lilbee does all of it in one install: it runs the models, processes your documents, crawls the web pages you point it at, and searches the lot with a real engine. The same engine works two ways:

• An Encarta 99 over your own files. Build a library from your documents and saved web pages, then read it and ask questions of it in the terminal.
• A reference layer for code. Point it at your project, dependencies, and API docs, and your coding agent answers from what's actually there, with file:line citations, instead of guessing function names.

The long-term goal: make local AI genuinely useful on hardware you already own, with no token budgets, no provider to depend on, and nothing about your files leaving your machine; the cloud's there only when you want it.

How lilbee compares

lilbee is built for consumer hardware and for people who don't want to babysit infrastructure. It isn't another model server you point an app at; it's a local search engine with the model runner built in. One install gives you the whole stack in a single executable:

• A search engine over your files, with answers that cite the source line, not just a model to chat with.
• A managed fleet, chat, embedding, vision, and reranking, spread across every GPU in the machine behind a load-balancing router, not one model loaded at a time.
• Everything bundled: model manager, search engine, web crawler, MCP server for coding agents (native opencode and hermes), HTTP server, TUI, and Python, in one file.

It sits between two worlds: the desktop runners that get a model chatting on your machine (Ollama, LM Studio), and vLLM, the server you stand up to push one model to a cluster of users. lilbee runs models to do retrieval over your files, and scales that whole stack across every GPU in the machine, from one small file.

Of the four, lilbee is the only one built around retrieval, and the only one that scales the whole stack, chat, embedding, vision, and reranking, across every GPU in the machine behind a load-balancing router.

Already on Ollama or LM Studio? lilbee runs on top of them. Prefer a GUI to the terminal? The Obsidian plugin maps lilbee's model manager and search to a visual interface inside your vault.

First start

The very first launch does one-time work: a bundled build unpacks itself behind a progress bar, and the model loads before your first answer, shown live inside the answer bubble. Every launch after that opens straight to chat in a couple of seconds.

Very first launch

Every launch after

Measured with a small chat model (Qwen3 0.6B):

Bigger models load longer; the bubble shows real progress while the weights load. The usage guide explains the engine lifecycle and how to keep the engine warm so relaunches skip the load.

What you can do with it

A library of your own files

Point lilbee at a folder of PDFs, notes, ebooks, or code and it builds a searchable library, with citations that click back to the source line. The pattern works for anything you have a lot of text about: a shelf of appliance manuals, a field's research papers, a car's service manuals, your company's internal wiki. Whatever you give it becomes searchable, and you can talk to it.

https://github.com/user-attachments/assets/73903a5c-9c0a-42a8-9ffd-bdf34f719c29

Ask it something with a real answer at stake and you get the answer, not a paraphrase of the question. Here it reads the towing section out of a car manual and answers with the page it came from, so you can check it. The panel on the right is the GPU doing it, on this machine.

https://github.com/user-attachments/assets/284c316c-85f8-46d1-9531-59476f0db9f7

Offline copies of websites

Install the [crawler] extra, point lilbee at a docs site, a wiki, or a vendor's API reference, and the pages get fetched, converted to markdown, and added to your library. From then on you can search or chat with that copy of the site offline, even after it changes or goes down.

https://github.com/user-attachments/assets/8380c49b-cc65-43ae-83d4-ba9e3aa84af4

Or crawl a whole site, not just one page. With recursive crawling on, lilbee follows the links and indexes the lot; watch the page count climb in the Task Center, then ask one question that synthesizes across the whole site.

https://github.com/user-attachments/assets/0c7fbed2-9b40-4976-9856-1720c4f9ffb0

Documents, code, and scanned images

lilbee splits indexing by what's being read:

• Prose and structured documents (PDFs, Office files, ebooks, HTML, 90+ formats) go through Xberg with heading-aware chunking, so each chunk keeps its section context.
• Code goes through tree-sitter's AST-aware splitter across 150+ languages, so chunks map to functions, classes, and modules instead of arbitrary line ranges.
• Scanned PDFs and photos go through OCR in 100+ languages: Tesseract for plain text (set LILBEE_OCR_LANGUAGE, e.g. eng+deu), or a local / remote vision model that keeps tables and layout as markdown.

Retrieval returns things that make sense on their own, not fragments cut through an argument or a function signature.

Pick and tune your models

You install and switch chat, embedding, vision, and reranking models from inside the terminal: browse the catalog, pull a model, pick a role. Retrieval and generation expose 50+ settings (chunk size, search strictness, reranker depth, and more), editable from the TUI, env vars, or a project-local config file. Sane defaults.

Tested GPUs and backends

Placement reads what the engine reports about your hardware, and every backend words that differently, so lilbee checks each backend on real silicon rather than inferring it from the last one. CUDA from an RTX 3090 up to H200, and up to eight A100s at once, Vulkan on NVIDIA and Intel, ROCm on an AMD Instinct MI300X, Metal on Apple Silicon, and a CPU-only host. docs/tested-gpus.md lists every machine and what each run settled. Send captures from hardware the table does not list.

Supported models

lilbee's engine is llama.cpp, so lilbee runs what llama.cpp runs: any GGUF model built on one of the architectures below. The list comes from the bundled engine itself and grows with every engine update.

One representative per family also runs the full pipeline end to end on real GPUs -- the models, method, and results are in docs/tested-models.md.

https://github.com/user-attachments/assets/993a63d4-9468-4a55-b9ed-e4b1ae1ed235

Already running Ollama or LM Studio? Keep them.

Watch it: Ollama as the model manager and LM Studio as the model manager. Point lilbee at a running manager, index a PDF on camera, and get a cited answer back.

lilbee works with Ollama and LM Studio. Finding and running models for you is the default and simplest path (lilbee pulls them and runs them on Metal / Vulkan / CUDA, no server to stand up), but you don't have to adopt a new model manager to use lilbee.

• Point it at a running manager. Your models in Ollama or LM Studio show up in the same catalog and role pickers (chat, embedding, vision, rerank), labeled by where they run, alongside lilbee's own and any cloud models. Mix freely.
• They stay read-only. lilbee lists and runs them but never pulls or deletes them, so their lifecycle stays in the app you already use.
• On pip / uv, this needs the [litellm] extra (pip install --pre 'lilbee[litellm]'); the Homebrew, AUR, Nix, Docker, Flatpak, and Snap builds already include it. See Install.

Both reels run chat and embedding on the other manager's models, picked out of the catalog's Library tab where each row names the server it runs on.

https://github.com/user-attachments/assets/52635a74-b69f-42df-b427-3c8f8499323f

https://github.com/user-attachments/assets/13ec8b12-f931-46ac-a500-8a7940d1a2bd

See when a model won't load before you download it

Hugging Face has thousands of GGUFs, but the bundled llama.cpp only supports a subset of architectures and brand-new ones take time to reach the pinned runtime. lilbee tags incompatible models in the catalog and refuses the download (with an override confirm), so you don't wait through a multi-GB pull only to hit "unsupported architecture" at load.

https://github.com/user-attachments/assets/b42effc3-a7f6-4391-904c-3ac897712172

Cloud models, when you want them

lilbee runs entirely on your machine by default. Two ways to use a cloud model when you want one:

• Bring your own key. Install the [litellm] extra, add an API key, then point any role (chat, embedding, vision, rerank) at a cloud model from the same catalog. The TUI shows a warning the whole time a cloud model is on.
• Pair lilbee with a cloud agent over MCP. Your files, the embeddings, and the index stay local. Any MCP-aware agent calls lilbee_search / lilbee_add and gets back cited snippets.

Either way, your files and the index stay on your computer. Only what you ask and the snippets needed to answer it get sent to the cloud model.

Agents

lilbee serves your coding agents two ways, both local: it runs the models they talk to, and it answers their searches with cited snippets from your library.

Launch your coding agent on local models

lilbee launch opencode, lilbee launch hermes, and lilbee launch claude set up lilbee's local models in your agent in one command. For opencode and hermes, lilbee registers itself as a provider and an MCP server in the agent's own config and leaves your existing setup intact; for Claude Code, lilbee serves an Anthropic-compatible API and points the session at it by env, without touching Claude Code's settings. Each launch warms a model and opens the agent pointed at it. No API keys, no provider setup, and nothing leaves your machine. Tool-calling works across many GGUF families; docs/agent-models.md has the verified list and how the QA harness measures it.

One model serves as many agents as you want to run. These reels show four working at once against a single local model, each in its own worktree, reading and searching this repository through lilbee.

https://github.com/user-attachments/assets/5f3fd1a4-f946-4008-bfd8-600142785327

Qwen3 Coder Next, 45GB across three RTX 4090s: four agents on one model, 138 tokens a second warm.

https://github.com/user-attachments/assets/510a1691-4289-4d98-b749-7522c1f28559

Gemma 4 26B is a thinking model, and lilbee streams its reasoning as visible text: every pane shows the model working through the problem before it answers.

https://github.com/user-attachments/assets/06dbf829-1e1e-4a1f-9ca2-32aa65086f13

Qwen3.6 27B on the same four tasks: each agent states what it expects, then reads the code to check itself.

https://github.com/user-attachments/assets/46ce5440-ee9d-4420-a881-031a1ccccec8

And the same setup scales up: Devstral 2 123B, 70GB across two A100 80GB cards, one agent at the model's full 14 tokens a second.

It tunes itself, too. Tell a small local model to widen lilbee's search when a first result comes back thin, and the second pass returns full function bodies with file:line citations; a more capable model does the same from a prompt like "improve your search results." The lilbee-mcp skill teaches your own model the pattern.

https://github.com/user-attachments/assets/7bfec600-4c06-4a2f-8f77-be7e92f4f33f

A reference for AI agents

Once configured, lilbee plugs into whatever agent you use, over MCP. Feed it your project's docs, your dependency source, your API docs, your design notes; the agent stops making up function names and instead reads the actual code, cites file and line, and says it doesn't know when the answer isn't in your library. The lilbee-mcp skill is the single entry point: drop it into .opencode/skills/ or .claude/skills/ and it documents every tool, the workflows the agent should follow, and points to drop-in AGENTS.md and worker-subagent starters under examples/agent-integration/. This works with cloud-model agents too; lilbee stays local, and only the queries and the returned chunks reach the model.

Your files, the search index, and the embeddings stay on your computer. The agent calls lilbee_search and gets back cited snippets with file and line. The agent reels above show this live: each pane reads and searches this repository through lilbee.

Run a model bigger than one card

When a chat model won't fit on a single GPU, lilbee spreads it across the ones you have. It sizes each role's memory with gguf-parser, keeps headroom on every card, and tensor-splits the chat model across the fewest GPUs that fit, with the embedder, reranker, and vision models placed alongside it behind a load-balancing router. This is automatic: ask a question and the model loads split across your cards, answering from your own indexed source. Here a 70B is spread across three consumer cards and answers a mechanic's question from an indexed car manual, citing the page it came from.

https://github.com/user-attachments/assets/2a2232e6-d8ce-4302-92e4-6467f4fb1bee

You can also place it by hand. The placement editor pins each role to the cards you choose, previews the fit before anything loads, and applies it live. Ask for a layout that can't fit and it tells you the exact shortfall instead of failing at load time.

TUI

lilbee (no args) launches a full Textual terminal app: streaming chat with clickable citations, a model bar with searchable pickers and a Search/Chat toggle, a Task Center for background jobs, and screens for the model catalog, settings, and the generated wiki. Type / for the command list; tab completion works everywhere.

https://github.com/user-attachments/assets/ab4c5847-2261-421d-91a0-e3838498901c

Ctrl+P opens the Textual command palette, ? on an empty prompt (or F1 anywhere) toggles the keybinding cheat sheet, /help opens the slash-command catalog. Every action lilbee can take is reachable from one of those three.

https://github.com/user-attachments/assets/7c4f1096-44a4-40b5-b511-0f6496c520c5

Conversations are kept. Reopen an earlier one and it comes back with its history and citations intact.

https://github.com/user-attachments/assets/3ee7fb50-b983-4761-807c-2f024bbc3756

Tell it something about yourself and it remembers, across conversations rather than only within one.

https://github.com/user-attachments/assets/08a69c31-d74f-4e0a-97b6-340dc195848c

Every setting is editable in the app, and the first run opens the model catalog to get you started.

https://github.com/user-attachments/assets/a578dfd1-61a5-4008-b9bf-e94539003b6e

https://github.com/user-attachments/assets/702975d8-5c1f-4fa7-92a6-bf594ef9d2e3

Every reel on this page (plus the extras that don't fit here) is at lilbee.sh/tutorial with long-form captions. Tape sources are in demos-src/ on the gh-pages branch. For commands and settings, see the usage guide.

HTTP Server

The HTTP server exposes a REST API any tool or GUI can hit: search (with SSE streaming), document lifecycle, crawling, model management, configuration. See the REST API reference and the usage guide for setup.

The Obsidian plugin is a GUI built on it: it starts the HTTP server in the background, and every citation opens a Source Preview scrolled to the exact spot. It is an official Obsidian community plugin: install it from Settings then Community plugins inside Obsidian. The plugin README has setup.

Running as a service (optional)

For tools that talk to lilbee's HTTP REST API (the Obsidian plugin, custom GUIs, anything hitting /api/*), your OS launcher can keep the HTTP server warm so requests skip the cold-start.

Supported formats

Document extraction powered by Xberg, code chunking by tree-sitter. lilbee handles every format Xberg can extract (100+) and tracks its list directly, so support grows as Xberg adds formats. The table below covers the common ones.

Wiki

lilbee reads the documents you've indexed and writes a wiki about them: one page per concept or entity, not per document. A subject that recurs earns its own page, written and cited from the source that mentions it most and cross-linked with [[wiki link]], with coverage across sources coming from synthesis pages. Every section is citation-verified against the source text before it publishes; lower-confidence pages wait in a drafts/ queue for review.

Pages are written on demand: open one that does not exist yet and lilbee writes it from the sources it has, then links it into the rest.

https://github.com/user-attachments/assets/3ba1478e-86b0-4a90-aba1-99384b4f92fb

See the usage guide for commands and configuration, and how the wiki is validated for the evidence behind it.

Other projects generate wiki-style pages too; they differ in what they read, where they run, and what happens between generation and publication.

Built on

lilbee stands on a stack of established open-source projects, all bundled into one install:

• Xberg parses 90+ document formats with heading-aware chunking.
• llama.cpp is the local model runtime: lilbee bundles its llama-server and starts it for you, so every chat, embedding, vision, and reranker call goes through it. llama-swap keeps a server per role resident together behind one endpoint, and gguf-parser estimates each model's memory footprint so lilbee loads what fits. Without llama.cpp there is no lilbee.
• Hugging Face Hub (via huggingface_hub) hosts the model catalog and handles every download. Search, browse, and pull all route through it.
• LanceDB is the embedded vector store.
• tree-sitter (via tree-sitter-language-pack) chunks code across 150+ languages.
• crawl4ai and Playwright crawl the web; Tesseract is the OCR fallback when no vision model is set.
• LiteLLM bridges cloud model providers (the [litellm] optional extra).
• Textual draws the terminal; Litestar runs the HTTP server.
• MCP Python SDK is the agent surface; Typer is the CLI; Pydantic is the config + validation backbone.
• Nuitka compiles the whole thing into the standalone single-file binary, bundling its own Python runtime so there is nothing to install and nothing to compile.

FAQ

Does my data leave my machine? No. Your files stay on disk and search runs locally. A cloud model is used only when you pick one.

Will a model fit my GPU? lilbee reads the GGUF file and your devices and estimates fit before you download, and splits large models across multiple GPUs. More at lilbee.sh/gpu.

Can my coding agent use it? Yes, over MCP. The agent reads your real code and docs before answering, cited to the file and line. More at lilbee.sh/mcp.

Support

Having trouble? See TROUBLESHOOTING.md for log locations and common failures.

lilbee is built and maintained by one person. If it is useful to you, you can chip in via PayPal. Bug reports and pull requests help just as much.

License

MIT. See LICENSE.