| Summary | Gives a coding agent a build → see → interact → debug loop over the native GUI app it is writing: launch the app, screenshot it, read its accessibility tree, inject mouse/keyboard/gestures, tail its logs, and diff frames to confirm a change landed. Drives apps as an external black box with no app integration, so any toolkit or language works. 30 tools with text-first responses (element ids, diffs, waits) so the agent need not screenshot every step, plus optional sandboxing of the launched app. Backends: X11, Wayland, Windows, macOS, Android emulators, and the iOS Simulator. | Windows desktop automation for LLM agents with entity-based actions instead of coordinate-only clicking. Uses UIA, CDP, screenshots, keyboard/mouse/clipboard, and terminal control, plus entity leases, verified delivery, causal context, and interaction memory to reduce silent UI automation failures. |
|---|