Claude Background Control

What Claude Background Control does

Claude Background Control connects Claude Code to the graphical user interface of a Mac. It can inspect application windows, identify controls, set text-field values, click controls, type text, send keyboard shortcuts, navigate menus, scroll, drag, capture screenshots, wait for interface changes, and launch or foreground applications.

The server is intended for OS Automation tasks where an agent needs to operate desktop software rather than call a web API. Accessibility actions are preferred because they can work with background applications without taking focus from the current task. Direct mouse and keyboard events are available when accessibility control is not sufficient, including for canvas-based or Electron content.

How it works

The package runs as an MCP server through Node and invokes built-in macOS tools including osascript, screencapture, and sips. It does not require native npm dependencies. The installation command registers a stdio server named computer-control with Claude Code:

claude mcp add computer-control -- npx -y claude-background-control

Most operations target the frontmost application when no app argument is supplied. An application process name can be provided to target a particular app. If several windows match a label or role, the frontmost window is selected; unresolved ambiguity produces IDs that can be used to choose a specific element.

inspect is the normal starting point. It reports element IDs, roles, labels, values, and frames, and can filter matches by role or label. Actions such as click_element and set_value use those accessibility details before falling back to physical input events.

Setup and configuration

The server requires macOS and Node 20 or newer. Development workflows that run the TypeScript tests directly require Node 22.18 or newer.

The application running Claude Code must be enabled under System Settings → Privacy & Security → Accessibility. This permission is needed by every tool except screenshot. Screen capture also requires Screen & System Audio Recording permission for that same application. Restart the application after changing permissions.

Permission errors identify the application and path that macOS expects. When Claude Code runs inside the Claude desktop app, the required path may be a versioned executable under ~/Library/Application Support/Claude/claude-code/; an update can therefore require granting access again.

Tools and capabilities

• inspect: enumerate accessible UI elements, optionally filtered by app, role, or label.
• click_element and set_value: activate controls or assign text-field values without moving the mouse when accessibility support is available.
• type and key: paste text, including IME input such as Korean, or send shortcuts such as cmd+s and return.
• wait_for: wait for a matching element to appear or disappear after an action.
• click, scroll, and drag: perform coordinate-based pointer operations.
• menu: list or select menu-bar items by path.
• screenshot: capture the main screen, a region, or an application's front window.
• activate_app: launch an application or bring it forward.

The type operation restores the previous clipboard contents after use, including when the clipboard contains images, files, or rich text.

Limitations and notes

Claude Background Control operates on the main display only. inspect does not descend into leaf-like roles such as buttons and static text, so some controls may require another action or coordinate-based interaction. With Stage Manager enabled, a background window may appear only as a side-strip thumbnail; screenshot(app) warns about this and recommends activating the application first. Direct coordinate actions also depend on the reported screen position and visible application state.