# TokenKnows [Health: Active]
**Category:** ๐ป Developer Tools
**Repository:** https://github.com/johnnywuj81/tokenknows
**GitHub Stars:** 4
**Views:** 0
**Installs:** 0
**Upvotes:** 0
**Directory Page:** https://allmcps.com/mcp/tokenknows
## Description
Distill AI coding sessions into weekly reports, ADRs and a knowledge graph - self-hosted
## Claude Desktop Quick Installation
Install path detected from listing signals. Uses `uvx` (confidence: high):
```json
"mcpServers": {
"tokenknows": {
"command": "uvx",
"args": ["config"]
}
}
```
## Documentation & README
TokenKnows
Distill AI coding sessions into living knowledge โ weekly reports, ADRs,
incident reviews, books, agent skills, and a knowledge graph.
English | ็ฎไฝไธญๆ
---
## What is TokenKnows?
You spend hours pair-programming with Claude Code, Codex, and Cursor. The decisions, bug hunts, and design trade-offs from those sessions evaporate the moment the terminal closes. TokenKnows captures them automatically and distills them into **structured, evidence-linked knowledge assets**:
**capture** (6 collectors) โ **distill** (5-stage LLM pipeline) โ **assets** (7 document types) โ **review / redact / publish**
- ๐ก **Captures everything** โ Claude Code, Codex, Cursor, VS Code, GitHub PRs/commits/issues, and local docs, all via local file watchers and API polling. No webhooks, no tunnels.
- ๐ **Seven asset types** โ weekly reports, tech designs, ADRs, incident reviews, long-form books, reusable agent skills (SKILL.md), and an entity knowledge graph.
- ๐ **Evidence-linked** โ every paragraph traces back to the original PR / conversation / commit, ranked by `cosine ร trust ร recency` across โฅ2 sources.
- ๐ **Local-first, zero egress by default** โ a three-layer LLM egress gate (instance โง project โง task) with full audit logging. Pair it with Ollama and run the whole pipeline with **zero cloud keys**.
## Demo
| Workbench | Document page |
|---|---|
| [](engineering_handoff/demo-screenshots/01-workbench.png) | [](engineering_handoff/demo-screenshots/04-document-page.png) |
| **Evidence drawer** | **Publish receipt + version diff** |
| [](engineering_handoff/demo-screenshots/05-evidence-drawer.png) | [](engineering_handoff/demo-screenshots/10-publish-receipt.png) |
โถ Full walkthrough: [`engineering_handoff/walkthrough.mp4`](https://github.com/johnnywuj81/tokenknows/blob/HEAD/engineering_handoff/walkthrough.mp4) (5 min, Chinese narration + subtitles)
All 12 screens
| 1 Workbench | 2 Event drawer | 3 Document list | 4 Document page |
|---|---|---|---|
| [](engineering_handoff/demo-screenshots/01-workbench.png) | [](engineering_handoff/demo-screenshots/02-event-drawer.png) | [](engineering_handoff/demo-screenshots/03-document-list.png) | [](engineering_handoff/demo-screenshots/04-document-page.png) |
| **5 Evidence drawer** | **6 Regenerate dialog** | **7 Review** | **8 Redaction** |
| [](engineering_handoff/demo-screenshots/05-evidence-drawer.png) | [](engineering_handoff/demo-screenshots/06-regenerate-dialog.png) | [](engineering_handoff/demo-screenshots/07-review-page.png) | [](engineering_handoff/demo-screenshots/08-redaction-page.png) |
| **9 Publish dialog** | **10 Publish receipt + diff** | **11 LLM egress** | **12 Admin** |
| [](engineering_handoff/demo-screenshots/09-publish-dialog.png) | [](engineering_handoff/demo-screenshots/10-publish-receipt.png) | [](engineering_handoff/demo-screenshots/11-settings-llm.png) | [](engineering_handoff/demo-screenshots/12-admin.png) |
## Install the plugin
Prerequisite: the TokenKnows backend at `http://localhost:8001` **and the web UI at `http://localhost:5173`** (see Quick start), plus [uv](https://docs.astral.sh/uv/) (the plugin pulls the MCP server from PyPI via `uvx`). All plugin env vars have working local defaults โ export `TOKENKNOWS_API_BASE` / `TOKENKNOWS_API_TOKEN` / `TOKENKNOWS_DEFAULT_PROJECT` / `TOKENKNOWS_WEB_BASE` only for non-default setups. Register/login in the web UI and create an API token under **Project Settings โ MCP ๆฅๅ
ฅ** when your backend requires auth.
| Platform | How |
|---|---|
| **Claude Code** | `/plugin marketplace add johnnywuj81/tokenknows` โ `/plugin install tokenknows@tokenknows` โ full walkthrough in [tokenknows-plugin/README.md](https://github.com/johnnywuj81/tokenknows/blob/HEAD/tokenknows-plugin/README.md) (5-minute quickstart) |
| **Codex** | `codex plugin marketplace add johnnywuj81/tokenknows` โ `codex plugin add tokenknows@tokenknows` (loads skills, commands and the MCP server; local-clone alternative in [codex-plugin/README.md](https://github.com/johnnywuj81/tokenknows/blob/HEAD/codex-plugin/README.md)) |
| **Cursor** | Add the tokenknows MCP block to `~/.cursor/mcp.json` (uvx config example in [code/tokenknows-mcp/README.md](https://github.com/johnnywuj81/tokenknows/blob/HEAD/code/tokenknows-mcp/README.md)) |
| **VS Code** | Download the `.vsix` from [Releases](https://github.com/johnnywuj81/tokenknows/releases) โ `code --install-extension tokenknows-vscode-*.vsix` |
The plugin gives your AI tool MCP tools (`submit_session_events`, `distill_document`, `list_assets`, `get_asset`, `get_asset_chapters`, `search_entity`) plus slash commands like `/tokenknows:weekly` and `/tokenknows:adr`.
## Quick start
```bash
# 1. (Optional but recommended) Ollama โ fully local inference, zero cloud keys
ollama serve &
ollama pull minimax-m2:cloud # or gpt-oss:20b, qwen2.5, ...
# 2. Backend (FastAPI + SQLite persistence + 3-layer LLM egress gate)
cd code/tokenknows-api
python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
cp .env.local.example .env.local # defaults to Ollama; edit to add cloud providers
.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8001
# 3. Frontend (React 19 + Vite)
cd code/tokenknows-web
npm install
npm run dev
# open http://localhost:5173 โ talks to the real backend (mocks are opt-in via ?msw=1)
# (Optional) seed demo data
./engineering_handoff/demo-seed.sh
```
**Platform support**: macOS โ full experience (collectors auto-start via launchd). Linux โ backend, frontend, and collectors all run manually (`python3 plugins//sync.py --watch`); the launchd scripts don't apply. Windows โ untested; WSL2 recommended.
## Data collectors
All local โ no ngrok, no public webhooks. On macOS they restart on crash and on reboot (launchd).
| Collector | Source | Mode |
|---|---|---|
| **claude-code** | `~/.claude/projects/*.jsonl` | 30s polling, incremental offsets |
| **codex** | `~/.codex/sessions/**/rollout-*.jsonl` | 30s polling, incremental offsets |
| **cursor** | Cursor's `state.vscdb` (read-only SQLite) | 60s polling |
| **github** | GitHub REST API ยท PRs / issues / commits | 5min polling (`gh auth` token) |
| **vscode** | VS Code extension `onDidSaveTextDocument` | buffered, 10s flush |
| **local-docs** | `~/Documents` `.md` `.txt` `.pdf` (watchdog) | realtime, 2s debounce |
```bash
./scripts/launchd/install.sh # macOS: install all 5 Python collectors as LaunchAgents
launchctl list | grep com.tokenknows
tail -f ~/Library/Logs/tokenknows/*.log
```
Every event carries a trust score (`0.6 ร source_authority + 0.4 ร extraction_confidence`); the evidence stage ranks citations by `0.6 ร cosine + 0.25 ร trust + 0.15 ร recency` and enforces โฅ2 distinct sources.
## Architecture
Collectors feed an event store (SQLite). A five-stage pipeline (collect โ outline โ content โ evidence โ assess) turns events into assets. The LLM Gateway unifies four providers (Anthropic / OpenAI / MiniMax / Ollama) with per-task routing and fallback chains โ and refuses any cloud call unless all three egress switches are on.
## CI
| Workflow | Runner | Trigger |
|---|---|---|
| [`ci.yml`](https://github.com/johnnywuj81/tokenknows/blob/HEAD/.github/workflows/ci.yml) | ubuntu-latest (GitHub-hosted) | push to main + every PR |
| [`ci-macos.yml`](https://github.com/johnnywuj81/tokenknows/blob/HEAD/.github/workflows/ci-macos.yml) | self-hosted macOS ARM64 | maintainer pushes to main only โ never runs external PR code |
## Privacy & local-first
- Zero egress by default โ cloud LLM calls require the instance **and** project **and** task switches all on
- Bring your own keys; the audit log never leaves your machine
- One-click kill switch drops the instance into fully-offline mode
Details: [PRD ยง6.7 data residency & egress control](https://github.com/johnnywuj81/tokenknows/blob/HEAD/docs/product/PRD_TokenKnows_MVP.md) (Chinese).
## Documentation
| Topic | Doc |
|---|---|
| Product requirements, user journeys | [PRD](https://github.com/johnnywuj81/tokenknows/blob/HEAD/docs/product/PRD_TokenKnows_MVP.md) (zh) |
| Technical design, API, schema | [TDD](https://github.com/johnnywuj81/tokenknows/blob/HEAD/docs/product/TDD_TokenKnows_MVP.md) (zh) |
| Macro architecture & milestones | [Architecture](https://github.com/johnnywuj81/tokenknows/blob/HEAD/engineering_handoff/Architecture.md) (zh) |
| Per-screen engineering decisions | [TaskTechDesign](https://github.com/johnnywuj81/tokenknows/blob/HEAD/engineering_handoff/TaskTechDesign.md) (zh) |
| Pixel-level UI mockups | [mockups/](https://github.com/johnnywuj81/tokenknows/blob/HEAD/mockups/) โ open in a browser |
> Most in-depth docs are in Chinese (the project's working language). Code comments are predominantly Chinese too; issues and PRs in English or Chinese are both welcome.
## Community
[CONTRIBUTING](https://github.com/johnnywuj81/tokenknows/blob/HEAD/CONTRIBUTING.md) ยท [Roadmap](https://github.com/johnnywuj81/tokenknows/blob/HEAD/ROADMAP.md) ยท [Code of Conduct](https://github.com/johnnywuj81/tokenknows/blob/HEAD/CODE_OF_CONDUCT.md) ยท [Security policy](https://github.com/johnnywuj81/tokenknows/blob/HEAD/SECURITY.md) ยท [Issues](https://github.com/johnnywuj81/tokenknows/issues)
## License
[MIT](https://github.com/johnnywuj81/tokenknows/blob/HEAD/LICENSE) ยฉ 2026 johnnywuj81