English | 简体中文

clawtouch-mcp

Give your AI agent real hands. An MCP server that turns any MCP-compatible client — Claude Desktop, Cline, Continue, Cursor, OpenClaw, Hermes Agent and any other — into something that can move a real mouse and press real keys through a USB HID device.

What is this?

A standalone Python process that speaks Model Context Protocol (MCP) over stdio and exposes mouse / keyboard primitives — hid.click, hid.type, hid.scroll, key combos, hid.screenshot — to whatever AI agent you already use. Under the hood it talks over USB serial to a ClawTouch HID device (a Raspberry Pi Pico 2 running the open ClawTouch HID firmware, or any turnkey ClawTouch box) and translates each tool call into a real USB HID report. The target OS sees a genuine physical keyboard and mouse — input arrives on the same driver path as any plugged-in peripheral, not as a software-injected synthetic event.

📦 MIT-licensed. No ClawTouch backend, no LLM, no agent loop on top — just the raw HID plumbing so other agent stacks can talk to real hardware.

⚠️ This gives an agent real keyboard / mouse reach over a machine — the same reach as a person at the keyboard. Read Safety first.

Why hardware HID?

Software automation (PyAutoGUI, OS-level input APIs, multimodal click-the-screen models) injects synthetic input events into a session — which requires an agent process running on the target machine, in that user session, with focus. A USB HID peripheral works the other way around: it emits real HID reports that travel the standard OS HID driver stack, exactly like a plugged-in keyboard or mouse. The OS recognizes the Pico natively as a standard USB HID class device and needs no mouse / keyboard driver and no HID agent process on the input side of the target. That difference is the whole point of this project — every other section below just builds on it.

Local mode is the common case (agent + clawtouch-mcp + Pico + the screen all on one PC; the clawtouch-mcp process lives there because it's the agent's host, but the input side needs no driver). Cross-host control — agent on one machine driving a target on another over USB HID — is an additional capability the same hardware unlocks; see Deployment modes.

Good for:

• Kiosks / locked-down machines — drive a machine you can't (or won't) install software on; nothing extra runs on the input side.
• Accessibility — let a user drive their own computer via an agent issuing HID commands, without fighting per-app synthetic-input compatibility.
• Compatibility testing — verify your software handles external HID input correctly, which can differ from injected synthetic events.
• Cross-host RPA / test rigs — an agent on your dev laptop drives an industrial PC, an offline test target, or a QA-lab phone, with no agent on the target (visual feedback needs a separate path — see Deployment modes).

Not for:

• Mass account creation / multi-account operations — a single-host tethered peripheral is structurally a poor fit; one device drives exactly one target, and to drive ten machines you buy ten devices.
• Application-specific scripted shortcuts (selectors, fixed-flow scripts for a particular site or app). Those belong in agent / RPA frameworks built on top of this primitive layer, not in this layer itself.

For standard desktop apps (browser, IDE, office suites) the software-only path is already enough — the hardware is just an extra option there, not a requirement. Its irreplaceable value is the cases above, where the target can't host an agent, must show the OS a genuine physical HID device, or has to be driven across machines. For the compliance boundary on the "not for" cases, see Acceptable use.

Quickstart

⚠️ Before you start: read Safety — a connected agent can operate this machine like a person at the keyboard.

Install

pip install clawtouch-mcp                     # minimal (serial only)
pip install 'clawtouch-mcp[screenshot]'       # + mss + Pillow (JPEG, Retina resize)
pip install 'clawtouch-mcp[screenshot-min]'   # mss only — no native deps, works
                                              # under hardened-runtime hosts
pip install 'clawtouch-mcp[window]'          # pyobjc — macOS needs it for
                                              # screen.windows; no-op elsewhere

Platform-specific setup guides (recommended on first install):

• Windows — docs/windows-setup.md: dual COM port enumeration, VS Code Claude extension .mcp.json config, full window restart required, display-scaling notes.
• macOS — docs/macos-setup.md: Keyboard Setup Assistant dialog on first plug-in, dual USB-CDC ports, Screen Recording permission, Pinyin IME punctuation gotchas.

Run

# 1. Auto-detect HID board AND auto-detect screen size (v0.2.3+)
clawtouch-mcp

# 2. Explicit port (Windows), screen still auto-detected
clawtouch-mcp --port COM7

# 3. Pin screen size manually (e.g. clamp to one monitor in a multi-monitor setup)
clawtouch-mcp --screen 1920x1080

# 4. No hardware — everything is logged, nothing moves (dev/CI mode)
clawtouch-mcp --mock --log-level INFO

v0.2.3+ auto-detects the primary monitor's physical pixel size on startup so coordinates clamp to the actual screen rather than a hard-coded 1920x1080. Use device.info from your MCP client to see what was detected (screen.source is "detected" / "explicit" / "unset").

Use with Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "clawtouch": {
      "command": "clawtouch-mcp",
      "args": ["--port", "COM7", "--screen", "1920x1080"]
    }
  }
}

Restart Claude Desktop. You should see clawtouch show up in the MCP server list with 16 tools available (14 HID + 2 device; +2 if you pass --allow-screenshot, which enables hid.screenshot and screen.windows). Try:

Take a screenshot of my screen, find the search box, click it, and type "hello world".

(Requires --allow-screenshot to enable the hid.screenshot tool — off by default for privacy.)

Other MCP clients

Copy-pasteable config for 7 verified clients (Claude Desktop / Code, Cursor, OpenClaw, Hermes Agent, ChatGPT Desktop / Codex CLI, Cherry Studio, Trae IDE) — see examples/integrations/INTEGRATIONS.md. PRs adding new clients welcome.

Deployment modes

Is the agent on the same machine as the screen? clawtouch-mcp covers the input side only (agent tool call → HID report → real input). The visual side (agent reads the screen to decide what to do next) is not in this repo — how you wire the two sides together depends on where the agent runs.

Local mode — the common case. agent + clawtouch-mcp + Pico + the controlled screen all on one PC. hid.screenshot captures that same screen, so the visual feedback loop closes naturally; the Pico is a standard USB HID device needing no driver. Good for accessibility, single-machine RPA, compatibility testing, in-machine kiosk self-service.

Cross-host mode — input supported, visual is your problem. agent + clawtouch-mcp on machine A; the Pico and the controlled screen on machine B. This repo fully covers the input side (A → B over USB HID), but hid.screenshot still captures A's screen, not B's — HID carries input one-way only; reverse screen capture isn't in the spec. Pick a visual path: HDMI capture card (B stays truly software-free, needs capture hardware) · VNC / RDP (open, no vendor lock-in, but B is no longer software-free) · API / log verification (check progress at checkpoints, not real-time; fixed-flow RPA only) · blind operation (pre-baked command sequence, no feedback; fully deterministic macros only). Good for industrial PCs that can't run a modern OS, strictly isolated embedded test targets, QA-lab phone farms.

Safety

Read this before connecting an autonomous agent. The runtime limits above are flood / typo guards, not a security boundary against a misbehaving agent.

Runtime safety limits