Ghost

Verified control of the whole desktop, for agents and programs, on Windows and Linux. Ghost gives Claude Code, Codex, Cursor, any MCP harness, or a plain script the operating system's own control surface: the apps with no API, the windows, the shell, and the browser you are already logged into - in the background without taking your screen or cursor, with every action proven to have happened.

Like Playwright, but for the native desktop, and built for agents. Ghost does the perceiving, the acting, and the verifying; the model you are already running does any looking that is needed, so there is no vision API key to set.

One MCP surface, two engines: Win32 UI Automation on Windows, AT-SPI2 over D-Bus on Linux. The verbs, the locator tiers and the act-then-verify loop are written once and behave the same on both. Platform support · Linux setup

Why Ghost is different

• Runs in the background, and that is enforced. An agent can click, type, and use shortcuts inside an app while you keep working in another window - no focus steal, no cursor jump. It posts window messages to real controls and uses UI Automation patterns on windowless ones; most tools can only drive whatever is in the foreground. Since 0.19 this is a policy, not a preference: the focus policy defaults to background. Since 0.20 it is also constructive: anything Ghost starts (an app, a windowed browser) is born on a hidden desktop that has its own input queue and cannot take your foreground, and the ordinary verbs drive it there by window title. A call that truly has no background path fails naming the action instead of quietly taking the screen. Since 0.22 the policy is locked there: no tool call can raise it, so an agent cannot decide on its own to take your mouse. You can, by setting GHOST_FOCUS_LOCK=off in the server's environment. And since 0.23 a window that grabs the foreground on its own - browsers do, on their own accessibility calls - is handed straight back, so your typing keeps going where you are looking. Measured on a real desktop with a person typing throughout: of roughly 490 keystrokes a run, zero to one reached the window the agent was driving. (how)
• Never your window by accident. The session remembers the last window the agent named or launched and every window-scoped verb targets it by default. The human's foreground window is used only when nothing was ever anchored, and the response says so. Three weeks of real transcripts showed "element not found in the foreground window" as the top failure before this; it was the agent searching the window you had open.
• Many agents at once. Requests are dispatched in parallel, so a 15-second wait in one tab does not stall an instant query behind it, and a second Ghost process runs its own browser alongside the first without contending for the mouse.
• Prove it on your machine. ghost verify drives the real MCP server over stdio and audits every claim above against hard timing budgets, exiting non-zero if any of them does not hold on your hardware.
• Every action is verified. Ghost re-checks the screen (or reads the control's value back) after acting and returns verified / focus_confirmed - never a blind ok:true. Agents fail by acting and not knowing if it worked; Ghost closes that loop.
• Drives apps with no API. Legacy Win32, WPF, Electron, UWP, vendor portals - the software that has no integration and most needs automating. No CDP, no browser, no app cooperation required.
• No vision key. The model driving Ghost reads ghost_see (every element with its name, role and on-screen centre) or ghost_screenshot itself. Ghost's own vision tier is optional, exists for callers with no model of their own, and is off until configured.
• Accessibility-native and deep. Real element discovery through the OS's own accessibility API - UI Automation on Windows, AT-SPI2 on Linux - not pixel-guessing. Elements come back with real names, roles and bounds.

See it in one script: examples/background_agent_demo.py drives an app in the background while the foreground stays yours. Honest comparison vs Playwright-MCP / cua-driver / Computer Use: docs/comparison.md.

What people use it for

Vision is the smallest part of it. In a typical session an agent calls Ghost mostly to act, to manage windows and processes, to run commands, and to read state back.

Every row runs under the same guarantees: the action returns verified, the default policy never takes your foreground, and Ctrl+Alt+G stops every Ghost process at once.

What is Ghost?

Ghost is the layer between a model and the desktop. The model reasons; Ghost sees the screen the way the operating system does, acts on real controls without touching your foreground, and reports whether the action took. It gives programmatic control over any desktop application - native Win32, Electron, WPF, UWP, GTK, Qt, or otherwise - to an agent, a script, or a program.

On Windows it uses UI Automation for element discovery, SendInput for keyboard/mouse injection, and DXGI/GDI for screen capture. On Linux it uses AT-SPI2 over D-Bus for discovery and actions, XTEST (X11) or the RemoteDesktop portal / uinput (Wayland) for input, and X11 GetImage or the Screenshot portal for capture. The Linux engine is pure Rust - no -devel packages to install.

Ship it three ways:

• ghost-mcp server - the primary surface: a Model Context Protocol server for Claude Code, Claude Desktop, Codex, Cursor, and any MCP client (54 tools on Windows)
• ghost CLI - one-shot commands, great for scripts and CI (ghost click --name "Submit")
• ghost-http server - local REST API, call it from Python, Node, curl, anything (curl http://127.0.0.1:7878/list-windows)

The MCP surface is 20 desktop verbs, 19 ghost_browser_* / ghost_tab_* tools for driving individual browser tabs in the background (Chrome, Comet, Edge, Brave), and 15 Windows-only tools: the focus policy plus ghost_desktop_* for explicit control of isolated Windows desktops the user never sees. Under the default policy you rarely need the latter: ghost_window op=launch already starts the app on the hidden desktop auto, and ghost_see / ghost_act / ghost_key / ghost_scroll reach it with window=<title> exactly as they reach a window on your own desktop (target.surface in the response tells you which). UIA, window messages and capture work fully there. Real SendInput does not, because Windows refuses it off the input desktop, and typing is proven by reading the control's value back, so a target that drops posted characters returns an error rather than a false success. The desktop verbs and the browser tools build on Linux as well; the focus policy and hidden desktops are Windows-only.

No Claude required. No browser required. No CDP. It drives apps through the OS's own automation and input APIs, so it works with native apps that have no API and no automation hooks of their own - the same reliability whether or not an app was built to be automated.

Platforms