Log inSubmit server
Claude Video Vision logo

Claude Video Vision

Analyze local videos and YouTube URLs with ffmpeg-extracted frames, audio transcription, scene detection, and configurable backends.

InstallRepositoryWebsite
Full README Compare alternatives Embed widget Claim this listing

Install Claude Video Vision

Pick your client, then add it in one click or paste the config.

STDIONode.js
~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "claude-video-vision": {
      "command": "npx",
      "args": [
        "-y",
        "claude-video-vision@latest"
      ]
    }
  }
}

Settings → Developer → Edit Config, merge this into mcpServers, then fully quit and reopen Claude Desktop.

Install not yet sandbox-tested— the server itself is confirmed live (we called its tools/list endpoint). The stdio install command is a separate, ongoing check.

Overview

claude-video-vision gives Claude Code a video perception layer for local files and YouTube URLs. It extracts timestamped frames with ffmpeg and processes audio through Gemini, local Whisper, or OpenAI, returning visual and transcription data for Claude to interpret. Use video_analyze before video_watch on videos longer than 30 seconds to identify scenes, silence, motion, and useful segments. The server fits workflows that need video summaries, screen-text inspection, tutorial analysis, or targeted review of selected moments.

Use cases

  • Summarize a long lecture or recorded presentation
  • Inspect on-screen text at a specific timestamp
  • Review a bug report or screen recording
  • Compare selected scenes from a local video
  • Transcribe and analyze a YouTube video

Key features

  • Extracts timestamped frames with ffmpeg
  • Processes audio with Gemini, local Whisper, or OpenAI
  • Analyzes scenes, silence intervals, and motion
  • Supports local files and YouTube URLs
  • Configures frame rate, resolution, format, and Whisper models
  • Caches and deduplicates frames with optional indexing

Tools6

What Claude Video Vision lets your agent do. Expand a tool for its parameters.

Verified live~2.2k tokens of context

Security & health

What it can reach and how reliably it runs — from our own checks, not the maintainer’s claims.

ActiveLast checked Today

Recent health check succeeded.

What it can do

From the 6 tools this server returned when we ran it.

Makes network requests · 4Read-only tools · 6
Network at startup3 connections
Files written at startupNone
Install scriptsNone
npm provenanceSigned build
Package age194 days
Maintainers1

Observed 2026-11-03 in an isolated sandbox. A description of reach, not a security audit.

Supply-chain signal

No high-severity advisories surfaced by our automated scan.

Critical 0High 0Medium 0Low 0

Scanned 16h ago via OSV.dev · claude-video-vision@latest (npm)

Adoption & maintenance

Factual signals from GitHub, npm, and our automated checks — not a rating.

GitHub stars1.5kStargazers on the source repository.
Last commit19d agoMost recent push to the default branch.
Tools exposed6Callable tools this server registers over MCP.

How Claude Video Vision works

What claude-video-vision does

claude-video-vision connects Claude Code to local video files and YouTube URLs. It supplies the visual and audio inputs needed for video understanding rather than generating the interpretation itself. Frames are returned as base64 images or text descriptions, while audio results can include timestamped transcription and audio-event analysis.

The server supports adaptive inspection. A request for a short moment can use a narrow time range and higher frame rate, while a long lecture can be sampled more sparsely. YouTube downloads use yt-dlp and can preserve source metadata and available captions.

How it works

ffmpeg handles video metadata, frame extraction, and structural analysis. Audio can be processed through one of three backends:

  • Gemini API, using GEMINI_API_KEY.
  • Local Whisper through whisper.cpp or Python openai-whisper.
  • OpenAI Whisper API, using OPENAI_API_KEY.

For videos longer than 30 seconds, run video_analyze first. Its results can identify scene changes, silence intervals, motion levels, and other structure so a later video_watch or video_detail call can choose more appropriate segments. video_analyze does not extract frames.

Longer audio may be split into chunks and transcribed in parallel. Results can include warnings about chunk boundaries, retries, or failed chunks. An incomplete analysis identifies the point where ffmpeg stopped; later portions should not be treated as empty.

YouTube audio results identify whether the transcription came from manual subtitles, automatic captions, or the configured audio backend. Manual English subtitles are preferred when available.

Setup and configuration

The server requires Node.js 24 or newer and ffmpeg. Install yt-dlp when YouTube URLs are needed. Backend-specific setup is required for cloud processing, while local processing requires Whisper tooling; models download automatically on first use. The setup wizard checks dependencies and guides backend and frame configuration.

Claude Code can install the plugin from its marketplace, after which the MCP package is fetched with npx on first use. Local development can instead run the plugin from a cloned repository. The /setup-video-vision command opens the configuration workflow, and /watch-video accepts a file or YouTube URL with an optional question.

Settings are stored in ~/.claude-video-vision/config.json. Options include audio backend, Whisper engine and model, frame mode and format, resolution, default frame rate, maximum frame count, frame-description model, indexing, and retention periods. Supported frame formats are JPEG, PNG, and WebP. Whisper models are stored under ~/.claude-video-vision/models/.

Tools and capabilities

  • video_info reads duration, resolution, codec, and other metadata without processing the video.
  • video_setup checks video-perception dependencies and provides installation guidance.
  • video_configure changes processing preferences.
  • video_analyze examines scene changes, silence, motion, and related structure without extracting frames.
  • video_watch combines frame extraction with audio processing and returns visual and audio results.
  • video_detail focuses extraction on selected segments, with variable frame rates and resolutions. Its view and view_sample controls determine which extracted frames are returned as images. Optional indexing caches and deduplicates frames across calls.

Limitations and notes

The server is a perception layer; Claude remains responsible for interpreting the returned frames and audio. YouTube support depends on yt-dlp and available captions or an enabled audio backend. Cloud backends require their corresponding API credentials and may incur usage costs; local Whisper avoids API usage but requires local installation and model resources.

For long videos, skipping structural analysis can lead to less informed segment selection. Audio warnings and incomplete-analysis details should be surfaced to users because they can affect transcription coverage.

Read the full README View source on GitHub

Reviews

No reviews yet — be the first to share how it worked for you.

Frequently Asked Questions about Claude Video Vision

claude-video-vision is an MCP server that connects Claude Code to local video files and YouTube URLs. Its main tools extract frames, inspect metadata, analyze scenes and silence, process audio transcription, configure preferences, check dependencies, and drill into selected video segments.

Install it through the Claude Code plugin marketplace, then let the server download through npx on first use. Add the marketplace URL, install the plugin, and run `/claude-video-vision:setup-video-vision`; Node.js 24+ and ffmpeg are required.

It supports Gemini API, local Whisper, and OpenAI Whisper API. Gemini requires `GEMINI_API_KEY`, OpenAI requires `OPENAI_API_KEY`, and local processing uses whisper.cpp or Python openai-whisper with models downloaded on first use.

Yes, it accepts YouTube URLs when yt-dlp is installed. The server prefers available manual English subtitles, then automatic captions, and falls back to the configured audio backend when captions are unavailable or insufficient.

Use `video_analyze` first for videos longer than 30 seconds. It identifies scene changes, silence, motion, and other structure without extracting frames, helping you select segments and frame rates before calling `video_watch`.

Related MCP servers

All in Multimedia Process All alternatives
Add a badge to your READMEShow it’s listed — live SVG, Markdown or HTML.

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewClaude Video Vision AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/claude-video-vision?style=directory)](https://allmcps.com/mcp/claude-video-vision)
HTML Embed
<a href="https://allmcps.com/mcp/claude-video-vision"><img src="https://allmcps.com/api/badge/claude-video-vision?style=directory" alt="Claude Video Vision on AllMCPs" /></a>