yt-mem-ai — a local YouTube memory for your AI assistant

Give Claude, Codex, Cursor, or any MCP host the ability to watch YouTube for you: transcribe videos, remember them, follow your subscriptions, and turn all of it into summaries, timestamped highlights, Q&A, digests, and video reels. Everything runs on your machine — no cloud service, no API key.

One command, two questions — here it's wiring the skills into Codex.

Example — "make a presentation from this video" (Andrej Karpathy: From Vibe Coding to Agentic Engineering, Sequoia, 30 min) → 13 slides, PDF, every quote timestamped from the transcript. Ingest to deck in one request.

Table of Contents

• Features
• How to install
• Getting Started
• Usage
• Examples
• Configuration
• Under the hood

Features

• 🎧 Transcribes any video — YouTube captions when they exist (fast, any language), offline Whisper when they don't.
• 🧠 Remembers what you watched — every transcript is stored and indexed locally, so your library stays searchable forever. Nothing leaves your machine.
• 🔎 Finds the moment — ask "what did that video say about X" and get the answer with a timestamp you can jump to.
• 📡 Follows your subscriptions — picks up new uploads and turns the day into one digest.
• ✍️ Your assistant does the writing — summaries, highlights, Q&A, slide decks, all in the video's own language, using the model you already pay for.
• ❤️ Learns your taste — like or dislike videos and get recommendations from your own library.
• 🎬 Makes media too — clickable highlight docs, still frames, and rendered supercut reels.
• 🔌 Works with your tools — Claude Code, Claude Desktop, Codex, Cursor, Antigravity, OpenClaw, Hermes: skills or MCP, your pick.

How to install

1. Connect your assistant ⭐

curl -LsSf https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/install.sh | sh

An interactive wizard opens. Pick what you want, tick your apps, press enter:

step 1/2 — what (pick one)     step 2/2 — where (tick any)
> Plugin  skills + CLI         [x] Claude Code    [ ] Claude Desktop
  MCP     typed tools          [x] Codex          [ ] Cursor
                               [ ] Antigravity    [ ] OpenClaw   [ ] Hermes

Plugin teaches your assistant to act on plain requests — "summarize this video". MCP gives it a set of tools instead. Not sure? Start with Plugin; you can run the wizard again for the other.

It installs everything it needs, ticks what you already have, and removes anything you untick (it shows a plan and asks first). Then restart the app and try: summarize 'https://youtu.be/…'.

Already know what you want? Skip the questions:

curl -LsSf https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/install.sh \
  | sh -s -- --plugin --claude-code --codex
curl -LsSf https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/install.sh \
  | sh -s -- --mcp --claude-desktop --cursor

Hosts: --claude-code --claude-desktop --codex --cursor --antigravity --openclaw --hermes, or --all. Full flag list and uninstall notes: integrations/README.md. Rather have an agent do it? Paste integrations/PROMPT.md into any assistant.

2. MCP by hand — one config entry, self-installing

No prior install needed: uvx fetches the package the first time the host launches the server, and keeps it cached afterwards. Drop this into your host's MCP config:

{
  "mcpServers": {
    "yt-mem-ai": {
      "command": "uvx",
      "args": ["--from", "yt-mem-ai[mcp]", "yt-ai-mcp"]
    }
  }
}

That's the whole setup — no paths, no env block. Settings live in ~/.yt-mem-ai/config.env and the agent can write them itself with the config_set tool (or you with yt-ai config set).

# ~/.codex/config.toml
[mcp_servers.yt-mem-ai]
command = "uvx"
args = ["--from", "yt-mem-ai[mcp]", "yt-ai-mcp"]

# ~/.hermes/config.yaml
mcp_servers:
  yt-mem-ai:
    command: "uvx"
    args: ["--from", "yt-mem-ai[mcp]", "yt-ai-mcp"]
    enabled: true

Restart the app and the tools show up — see Usage for what they do.

Nothing appeared, or the host timed out? The first launch downloads dependencies and can outlast the host's startup check. Run uvx --from 'yt-mem-ai[mcp]' yt-ai-mcp --help once, then reopen the app. If the host still can't start it, give it absolute paths — uv tool install 'yt-mem-ai[mcp]' and use which yt-ai-mcp as command with "args": [] (GUI apps often don't see ~/.local/bin on their PATH).

3. Claude Desktop — skills (in the app)

Desktop stores plugins on your Claude account, not on disk, so nothing can install them for you. It takes a minute in the app:

Customize (left sidebar) → Plugins → Personal plugins → + → Add marketplace → Add from a repository → https://github.com/dasein108/yt-mem-ai → Add → Install yt-mem-ai

Then ask: summarize 'https://youtu.be/…'. Uninstall the same way. The same plugin also works on claude.ai and Cowork. Prefer tools over skills? The MCP setup above works for Desktop too — and that one can be scripted.

# from a checkout
cp -R skills/yt skills/yt-agent ~/.codex/skills/

# without a checkout
for s in yt yt-agent; do
  mkdir -p ~/.codex/skills/$s
  curl -LsSf "https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/skills/$s/SKILL.md" \
    -o ~/.codex/skills/$s/SKILL.md
done

4. The CLI on its own

The skills drive it, but it's a perfectly good standalone tool:

uvx yt-mem-ai --help          # zero-install run
uv tool install yt-mem-ai     # or install the persistent `yt-ai` command

Needs Python 3.11+ and uv; ffmpeg only for supercut / frame.

The desktop UI lives in a separate repo: yt-mem-ai-desktop — it depends on this package and runs its own local REST API.

Getting Started

Installed and host restarted? You're ready. Just talk to your assistant — the skills (or MCP prompts + analyze_video) do the ingesting for you:

"Summarize https://youtu.be/dQw4w9WgXcQ" → ingests the video (captions → whisper), then writes an executive summary plus key points, in the video's own language.

"Give me the highlights of that video with timestamps" → 3–8 deep-linked moments (watch?v=…&t=123s) anchored by semantic search.

"What did I watch about retrieval-augmented generation?" → searches every transcript in your library and quotes the moments.

"Process my subscriptions into today's digest" → discovers new uploads, ingests them, writes digests/<DATE>.md.

Prefer the terminal? The same first run:

yt-ai fetch 'https://www.youtube.com/watch?v=VIDEO_ID'   # ingest one video
yt-ai search "what was said about embeddings"            # search your library
yt-ai status                                             # what's in the store

Everything lands in ~/.yt-mem-ai/ (library, logs, downloads).

The first run is slow — that's expected. Installing pulls the ML stack (torch, LanceDB, sentence-transformers ≈ 1 GB on disk), and your first fetch or search downloads the embedding model on top of that. If a video has no captions, the Whisper model (small, ≈ 460 MB) downloads too — the captions path never needs it. All of it is cached, so it happens once, not per video. On a GUI host the first MCP launch can outlast the app's startup check for the same reason: run uvx --from 'yt-mem-ai[mcp]' yt-ai-mcp --help once in a terminal to warm the cache, then reopen the app.

Daily routine