# screenhand [Health: Active]

**Category:** 🖥️ OS Automation  
**Repository:** https://github.com/manushi4/screenhand  
**GitHub Stars:** 15  
**npm Downloads (last month):** 366  
**Views:** 1  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/screenhand

## Description
Give AI eyes and hands on your desktop. ScreenHand is an open-source MCP server that lets Claude and other AI agents see your screen, click buttons, type text, and control any app on macOS and Windows.

## Claude Desktop Quick Installation
Install path detected from listing signals. Uses `npx` (confidence: high):

```json
"mcpServers": {
  "screenhand": {
    "command": "npx",
    "args": ["-y","screenhand"]
  }
}
```

## Documentation

## What screenhand MCP server does

The screenhand MCP server connects MCP-compatible AI clients to desktop applications on macOS and Windows. It can inspect native interface trees, activate applications, press controls, type text, navigate menus, drag, scroll, and send keyboard input through operating-system accessibility APIs. The server is intended for workflows that cross application boundaries, such as extracting information from Chrome and entering it into a desktop app.

Browser automation uses Chrome DevTools Protocol. Available browser operations include navigation, tab access, JavaScript execution, clicking, typing, and form filling. The server also includes fallback actions that can try accessibility APIs, CDP, OCR, and coordinates when one interaction method is insufficient.

Beyond direct interaction, the tool set includes screen perception, goal-oriented planning, recovery, application maps, learning records, and background jobs. Jobs can run multi-step tasks, detect stalls, recover, and coordinate agents with session leases. The installation includes reference material, playbooks, and app maps for selected applications and websites.

## How it works

The screenhand MCP server runs as a standard stdio MCP process. An MCP client sends tool calls, and the server performs actions on the local computer. Native desktop actions use accessibility APIs rather than requiring a screenshot and coordinate estimate for every click. Chrome actions use CDP and can operate in the browser's background when remote debugging is enabled.

The server records information from normal usage to build per-application maps. These maps can contain navigation paths, interface hierarchy, visible elements, timing information, state information, and readiness signals. Stored maps are located under `~/.screenhand/app-maps/`. Its learning features also record tool calls, strategies, error patterns, and recovery information.

## Setup and configuration

Install or run the package with `npx -y screenhand`, then register that command in an MCP client. The README provides configurations for Claude Desktop, Cursor, Claude Code, and OpenAI Codex CLI. It also describes ScreenHand as compatible with any MCP client that supports stdio.

On macOS, grant Accessibility access to the terminal application running the server under System Settings > Privacy & Security > Accessibility. Windows does not require special permissions according to the project documentation.

Browser automation is optional. To enable it, launch Chrome with remote debugging on port 9222. The project also provides a `--info` command for checking an installation. Building from source requires Node package installation and a platform-specific native build command.

## Tools and capabilities

The documented tool groups include:

- Desktop control for native applications.
- Browser control through Chrome DevTools Protocol.
- Fallback interaction across accessibility, CDP, OCR, and coordinates.
- Memory, strategy learning, and application mastery maps.
- Feature discovery from official application websites.
- Background jobs, orchestration, leases, stall detection, and recovery.
- Perception loops, entity tracking, planning, and self-healing recovery.

The README lists 111 tools in total and identifies prebuilt knowledge for 36 apps, including references, playbooks, and app maps. Exact coverage depends on the application and the available interface methods.

## Limitations and notes

Desktop control is limited to macOS and Windows. macOS use depends on Accessibility permission, and browser tools depend on Chrome being started with remote debugging. The project describes its prebuilt knowledge as covering selected apps and websites rather than every application. Building from source uses different native build commands on macOS and Windows. The package is licensed under AGPL-3.0.

_Full upstream README: https://allmcps.com/mcp/screenhand/readme_