Screencast MCP
An MCP server that lets an agent record the screen, watch the footage, and make minimal ffmpeg edits — over stdio, on Windows.
Capture the desktop, a monitor, a window, or a region; sample a recording into frames the agent can actually look at; trim, crop, redact, annotate, zoom toward clicks, and export for GitHub or social. No cloud service, no telemetry: ffmpeg and ffprobe behind 59 typed tools.
For developers who want their coding agent to see and prove UI changes on Windows: bug repros, UI verification, and demo clips the agent records, checks, and edits itself. Capture is Windows-only; the watch and edit tools work on any OS with ffmpeg. Free for non-commercial use (CC BY-NC-ND 4.0); why not OBS, Playwright, or a screenshot MCP?
Made with the tool: start_recording with events: true, one click, then zoom { highlightClicks: true } and export_gif. How it was made · more workflows
Quick start · Tools · Configuration · More docs · Privacy
Quick start
The fastest path: npx -y @tmhs/screencast-mcp setup checks FFmpeg 8, OCR, and speech, offers winget install Gyan.FFmpeg when FFmpeg is missing, and adds the server to Claude Desktop, Claude Code, Cursor, and VS Code (backing up each config). --dry-run shows the changes first; --test records a 3-second GIF to prove it works. Or set it up by hand:
You need Windows 10 or 11, Node.js 22+, and FFmpeg 8 or newer on PATH (winget install Gyan.FFmpeg, the full build). Older builds run most tools, but transcribe (the whisper filter) and backend: "wgc" (gfxcapture) need FFmpeg 8, and essentials or distro builds can lack them (Ubuntu LTS apt install ffmpeg gives 6.x). Capture is Windows-only; the watch, edit, and produce tools work anywhere ffmpeg runs. After connecting, ask the agent to run doctor: it reports the ffmpeg version, the filters and devices it has, OCR, and anything missing.
Windows on ARM64 is in progress (#246): CI runs a non-blocking windows-11-arm job. gyan.dev ships no arm64 FFmpeg, so use BtbN's winarm64-gpl FFmpeg 8 build; it has gdigrab, gfxcapture and drawtext but no whisper or libvpx, so transcribe and .webm output need another build. doctor reports arch and what is missing.
Add it to your client:
| Client | How |
|---|---|
| Claude Code | claude mcp add screencast -- cmd /c npx -y @tmhs/screencast-mcp, or the plugin (the server plus bug-repro, README-demo, and UI-verify skills): /plugin install screencast --marketplace TMHSDigital/screencast-mcp (older Claude Code: /plugin marketplace add TMHSDigital/screencast-mcp, then /plugin install screencast@screencast-mcp) |
| Claude Desktop | Download screencast-mcp.mcpb from the latest release and open it (one click) |
| Cursor, VS Code, others | The JSON config below, or step-by-step on Connect your client |
{ "mcpServers": { "screencast": { "command": "npx", "args": ["-y", "@tmhs/screencast-mcp"] } } }
Then ask your agent:
Record my VS Code window for five seconds while I reproduce the bug, then show me the frames where something changed.
More in the workflows: a bug-repro GIF for a PR, a README demo, a UI verification loop, a social clip, a cheap review of a long recording, zoom toward clicks, and the whole edit in one call.
Why not just...
The core is the watch loop: the agent records, samples the footage into frames it can see, and checks what actually happened on screen. Capture only ever happens on an explicit tool call, and output lands under SCREENCAST_HOME, never in your checkout.
| Instead of | The difference |
|---|---|
| OBS, ShareX, Game Bar | Built for a person at the controls. Here the agent starts and stops the recording, looks at the frames, and makes the edits through typed tools, with no GUI step. |
| Playwright or Puppeteer video | Browser pages only, and the video is a file the model never sees. Here: any window or monitor, frames returned to the model, plus wait_for_change, find_text, and screen_diff for checking what happened. |
| A screenshot-only MCP server | Single stills. Here: video over time, an input event log, scene-change sampling, redaction, and export, so the agent can show a sequence, not one moment. |
Tools
59 tools in 6 toolsets, generated from the server's own schemas by npm run docs (do not edit between the markers). Each name links to its reference page with every parameter, an example call, and what it returns; mcp-tools.json is the same list as JSON, and llms-full.txt all of it in one file.
Core
Always loaded, whatever SCREENCAST_TOOLSETS selects.
| Tool | What it does |
|---|---|
get_media_info | Probe a media file with ffprobe and return duration, resolution, frame rate, codecs, container format, and size. |
doctor | Check the setup: ffmpeg/ffprobe (FFmpeg 8+?), optional filters, capture devices, Windows OCR, settings, and enabled toolsets; problems says what is missing and how to fix it. Call on first use or when a tool reports a missing feature. |
Capture
Record or screenshot the desktop, a monitor, a window, or a region. Every capture is an explicit tool call: nothing records on its own.
| Tool | What it does |
|---|---|
start_recording | Start a background screen recording. Returns a session id; finalize it with stop_recording. Never auto-starts. |
stop_recording | Stop a recording gracefully so the file is finalized; returns the path and duration. |
delete_recording | Delete a finished recording with its sidecars (event log, webcam, transcript, index, markers, pause parts) and registry entries; reports bytes freed. Asks the user first when the client supports it. Edits made from it are kept. |
pause_recording | Pause a recording; until resume_recording nothing is captured, and the gap is left out of the file, events, and markers. |
resume_recording | Resume a paused recording (same target and settings). |
save_replay | Save the last N seconds of a replay buffer (start_recording replayBufferSec) to an mp4; the buffer keeps running. |
list_sessions | List all recording sessions (active and finished) with their status, target, output path, and whether the process is still alive. |
get_session | Inspect a single recording session by id, including live status and whether its output file exists on disk. |
read_events | Read the input events (clicks, scroll, cursor, shortcuts, typing bursts) of a recording made with events: true, in video seconds and pixels; e.g. types: ['down'], then sample_frames there. |
add_marker | Mark a labelled moment on a recording, live or at a second. Kept in the session and a sidecar; chapters and pipeline turn markers into chapters. |
screenshot | Capture one PNG of the screen, a monitor, a window, or a region, returned inline and by path. |
list_audio_devices | List DirectShow audio devices and flag a likely loopback (Stereo Mix or a virtual-audio driver), for start_recording's audio.device. |
list_video_devices | List DirectShow cameras for start_recording's webcam.device. |
list_targets | List the desktop, monitors, and visible windows (topmost first, with capturability per backend). Each entry has a target string to pass as-is to capture tools; use it instead of guessing a title. |
Watch
Turn footage into something a model can read: frames, contact sheets, diffs, OCR, and a searchable text timeline.
| Tool | What it does |
|---|---|
review_recording | Open a recording for a person to review: in MCP Apps clients an inline player to scrub, trim, and redact, then export; elsewhere just the video's details. |
player_frames | Preview frames for the review player: JPEGs on an even grid over the video plus frames at scene changes, with their real times. Used by the inline player; agents should use sample_frames or contact_sheet instead. (app-only: called by the review player, not the agent) |
sample_frames | Sample PNG frames from a video (by fps, timestamps, or scene changes) and return the first few inline so the model can see what happened. |
contact_sheet | Tile timestamped frames of a video into one image to see a whole recording at a glance (the cheapest first look). |
screen_diff | Compare two screenshots or frames: SSIM score, changed share, and changed boxes in b's pixels, to check a UI change happened only where expected. |
wait_for_change | Wait until part of the screen changes or settles, instead of guessing a sleep. Returns the elapsed time, changed regions (screenshot pixels; add origin for desktop), and the final state. |
find_text | OCR (built into Windows) on the screen, an image, or video frames, with line and match boxes in image pixels (add origin for desktop coordinates). Locate UI text or secrets (e.g. regex 'sk-|ghp_|api.?key') to redact. Best-effort: review results. |
index_recording | Build a recording's text timeline (.index.json: OCR per scene, events, markers, transcript) for search_recording. |
search_recording | Search an index_recording timeline (screen text, events, markers, transcript): timecoded hits, best first, secrets masked. No frames unless asked. |
Edit
Cut, convert, compress, and redact. A tool that writes a file refuses to replace an existing output unless overwrite: true; default output paths are always unique.
| Tool | What it does |
|---|---|
trim | Cut a video down to one sub-clip by start + (end or duration). For several segments use clip; to combine with other edits use pipeline. |
concat | Join videos end to end with hard cuts: stream copy when they match, else a re-encode (mode says which). For transitions use xfade_transition or assemble_highlights. |
convert | Convert media to mp4, webm, or gif. For a size-bounded GIF/WebP prefer export_gif; for a smaller mp4, compress or export_preset. |
export_gif | Export an animated GIF or WebP that fits a size budget (for PRs, issues, READMEs), stepping fps, width, and quality down until it fits. |
crop | Crop a video to a pixel rectangle; one running off the frame is rejected. |
scale | Resize a video to a target width and/or height and re-encode. Provide one side to keep the aspect ratio, or both to set an exact size. |
speed | Change playback speed by a factor (>1 faster, <1 slower) and re-encode. Audio is retempo'd to match when the source has an audio track. |
compress | Re-encode a video smaller (H.264 CRF ladder, optional width cap). For a platform target use export_preset. |
clip | Extract several frame-exact segments, each to its own file. For one segment, trim does the same. |
cut_idle | Remove dead air: cut stretches where the screen is frozen (and silent), keeping a short pad. Returns the time saved. |
cut_by_transcript | Edit a video by its transcript: cut quoted text, cue numbers, or fillers. Also writes the re-timed .srt. |
redact_region | Cover given rectangles in a video or screenshot (e.g. secrets on screen). Not automatic detection: find them with find_text redact: true. Off-frame regions are rejected. |
pipeline | Run several edits (trim, crop, scale, speed, cut_idle, redact, callout, keystroke_overlay, cursor, reframe, zoom, backdrop, audio, burn_subtitles, then optionally export_preset or export_gif) in one ffmpeg pass: one re-encode, no intermediate files. Prefer it over chaining edit tools; a recipe does the usual polish in one call. |
Produce
Annotate, frame, and assemble a finished piece. Tools that combine clips normalize each input to a common resolution, fps, and audio rate first.
| Tool | What it does |
|---|---|
overlay | Composite an image or video (logo, watermark, picture-in-picture) onto a video at a pixel position, optionally scaled and time-limited; keeps the base's audio and length. |
callout | Annotate a video or screenshot with timed boxes, text labels, and arrows. Fallback fonts used are listed in fallbackFonts; characters no font covers are reported in warnings. |
keystroke_overlay | Burn keycast-style badges for shortcuts, clicks, and typing from the recording's input event log (start_recording events: true). Never shows typed text: the log has none. |
cursor | Draw a smoothed, scaled cursor that hides when idle from the event log of a start_recording cursor: "events" recording (no OS cursor in the video), with click ripples and an optional spotlight. |
backdrop | Frame a video or screenshot on a gradient, color, or wallpaper with padding, rounded corners, and a drop shadow, optionally on a 16:9/1:1/9:16/4:5 canvas. |
burn_subtitles | Burn SRT/ASS subtitles (e.g. from transcribe) into a video in the bundled font. |
chapters | Write chapters (markers, scene changes, transcript, or a list) into an mp4/mkv/mov copy without re-encoding; returns YouTube chapter text. Can mux an SRT as soft subtitles. |
xfade_transition | Join two videos with an xfade transition (audio crossfaded); they need not match in size, fps, or audio. For more clips use assemble_highlights. |
assemble_highlights | Stitch clips (including title cards) into one video with cuts or xfade transitions; they need not match in size, fps, or audio. |
title_card | Generate a title card clip (centered text on a solid background, silent audio track) to stitch in with assemble_highlights. Uncoverable characters are reported in warnings. |
reframe | Re-aspect a video to 16:9, 9:16, 1:1, or 4:5 by padding or cropping. |
zoom | Smooth animated zoom-and-pan into the action and back out, from keyframes or planned from logged clicks or screen changes. |
export_preset | Encode a platform-ready H.264 file (aspect, fps, bitrate, faststart) for a named platform. |
bug_report | Package a recording as a paste-ready repro folder: secret-scanned, redacted clip and GIF, contact sheet, steps from the event log, OCR'd errors, environment, report.md for gh issue create --body-file. |
step_guide | Turn a recording's event log into a numbered how-to: a ringed screenshot and caption per step, marker headings; Markdown and/or one HTML file. Never includes typed text. |
Audio
Extract, clean up, level, transcribe, and score the soundtrack.
| Tool | What it does |
|---|---|
extract_audio | Extract the audio track of a media file to its own file (mp3, aac, wav, or copy). Returns the output path. |
normalize_audio | Normalize loudness to a delivery target (two-pass EBU R128 loudnorm); video is copied. Reports loudness before and after. |
denoise | Reduce steady background noise (fan, hum, hiss) in a recording's audio with an adaptive FFT denoiser. Video is copied untouched. Returns the output path. |
transcribe | Transcribe a recording's speech to an SRT subtitle file, locally, with FFmpeg 8's whisper filter (needs a whisper.cpp model file). Returns the SRT path and the text. Pair with burn_subtitles to caption the video. |
music_bed | Lay music under a video: looped or trimmed to length, faded, leveled, and mixed with any existing audio. Video is stream-copied. |
narrate | Offline voiceover with the built-in Windows voices: speaks each line at its time or marker over the audio, plus an SRT. |
[!IMPORTANT]
redact_regioncovers only the rectangles you declare; it is not automatic secret detection (usefind_textwithredact: trueto locate them). Prefer the defaultboxstyle for real secrets: it is a solid, irreversible fill, whileblurandpixelatecan be partially recovered.
Every capture tool takes one target string: full, monitor:<index>, window:<title>, window:exe=<app.exe>, window:hwnd=<handle>, or region:<x>,<y>,<w>,<h>; list_targets returns ready-made ones. See start_recording and the Windows notes for how windows, monitors, and backend: "wgc" are captured.
Configuration
ffmpeg and ffprobe must be on PATH, or set FFMPEG_PATH / FFPROBE_PATH. All settings are optional environment variables; doctor reports the ones in effect.
| Variable | Default | Purpose |
|---|---|---|
SCREENCAST_HOME | <homedir>/.screencast-mcp | Where recordings, frames, screenshots, and edits are written |
SCREENCAST_ALLOWED_DIRS | none | ;-separated folders tools may also read from and write to; when set, or when the client sends roots, files outside SCREENCAST_HOME, the roots, and these folders are refused. Without either, replacing an existing file outside SCREENCAST_HOME is refused. Network (UNC) paths are always refused |
SCREENCAST_MAX_RECORDING_SEC | 3600 | Default recording cap (0 disables it) |
SCREENCAST_JOB_TIMEOUT_MS | scales with length | Override the edit job timeout |
SCREENCAST_WHISPER_MODEL | none | whisper.cpp model for transcribe |
SCREENCAST_RETENTION_DAYS | off | Delete screenshots, frames, leftover raw event logs, and index_recording indexes older than this many days at startup (never recordings or edits; an index is rebuilt by running index_recording again) |
SCREENCAST_PS_TIMEOUT_MS | 20000 (OCR 60000) | Limit for the PowerShell helpers that list monitors and windows and run OCR |
SCREENCAST_CONFIRM_CAPTURE | off | Set to 1 to ask the user to confirm every live capture through MCP elicitation; clients without elicitation are refused |
SCREENCAST_TOOLSETS | all | Comma-separated toolsets to expose; the rest are not loaded, which shrinks the tool list the agent pays for on every turn (core is always on) |
FFMPEG_PATH / FFPROBE_PATH | PATH lookup | Explicit ffmpeg / ffprobe binaries |
| Toolset | Tools |
|---|---|
capture | start_recording, stop_recording, delete_recording, pause_recording, resume_recording, save_replay, list_sessions, get_session, read_events, add_marker, screenshot, list_audio_devices, list_video_devices, list_targets |
watch | review_recording, player_frames, sample_frames, contact_sheet, screen_diff, wait_for_change, find_text, index_recording, search_recording |
edit | trim, concat, convert, export_gif, crop, scale, speed, compress, clip, cut_idle, cut_by_transcript, redact_region, pipeline |
produce | overlay, callout, keystroke_overlay, cursor, backdrop, burn_subtitles, chapters, xfade_transition, assemble_highlights, title_card, reframe, zoom, export_preset, bug_report, step_guide |
audio | extract_audio, normalize_audio, denoise, transcribe, music_bed, narrate |
core (always) | get_media_info, doctor |
More docs
-
Usage: workflows walk through real tool calls and outputs: a bug-repro clip, a README demo, a UI verification loop, a social clip, a long-recording review, zoom toward clicks, and the whole edit in one
pipelinecall. Each tool's parameters and results are on its reference page. -
Command line and CI: the same package is a headless CLI (
npx -y @tmhs/screencast-mcp record -- npm run test:e2e,... call doctor), and a GitHub Action records your test run onwindows-latestand uploads the evidence on failure (being published to GitHub Marketplace). See the CI page:- uses: TMHSDigital/screencast-mcp@v0 with: command: npm run test:e2e -
Prompts and resources: the server also ships MCP prompts that clients show as slash commands (
bug-repro-clip,file-bug-report,document-steps,demo-gif,verify-ui-change,review-recording) andscreencast://sessionsresources for attaching recordings without a filesystem tool. A client can subscribe to the list or one session and is notified when it changes, such as a recording that stopped at its cap. -
Threat model and Windows notes: what capture can expose, and how window, monitor, WGC, and system-audio capture behave, are on the docs home page.
Privacy
- No network requests, no telemetry. The server runs ffmpeg, ffprobe, and PowerShell locally and talks to your client over stdio. OCR (
find_text) and speech (narrate) use the engines built into Windows. - What a tool returns goes to your model provider. Inline frames, OCR text, and event summaries reach your agent like any tool result; files on disk stay on disk unless a tool returns them or you share them.
- Everything is stored under
SCREENCAST_HOME(recordings/,screenshots/,frames/,edits/,sessions.json) unless a call names another path.find_textandwait_for_changedelete their captures after the call by default (keep: truekeeps them);screen_diffcompares existing images and captures nothing. - The event log never records typed text. Only with
start_recordingevents: true(orcursor: "events"), and only while that recording runs: clicks, scrolls, and cursor positions system-wide, and typing as bursts on a 0.5 s grid with no key names (keys: "shortcuts"also names Ctrl/Alt/Win chords). There is no microphone capture; system audio only withaudio.source=system. - Deleting data.
delete_recordingremoves a recording with its sidecars (event log, webcam, transcript, index with its screen text, markers) and registry entry, asking you first when the client supports it. Otherwise delete files underSCREENCAST_HOME, or setSCREENCAST_RETENTION_DAYS; recordings and edits are never deleted automatically.
Contributing
Issues and pull requests are welcome — see CONTRIBUTING.md (development setup and project structure) and the Code of Conduct. Security reports go through SECURITY.md.
License
Released under the CC-BY-NC-ND-4.0 license: free to use and share for non-commercial purposes, with attribution and without modification. For other uses, open an issue.
Can I use it at work? Only for non-commercial purposes under this license. If your use is commercial (inside a paid product or service, or work done mainly for commercial advantage), or you are not sure, open an issue to ask about a commercial license. The licensing model is described in the TMHSDigital licensing standard.