# fixed-width/glass [Health: Active]

**Category:** 🖥️ OS Automation  
**Repository:** https://github.com/fixed-width/glass  
**GitHub Stars:** 13  
**Views:** 2  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/fixed-width-glass

## Description
Gives a coding agent a build → see → interact → debug loop over the native GUI app it is writing: launch the app, screenshot it, read its accessibility tree, inject mouse/keyboard/gestures, tail its logs, and diff frames to confirm a change landed. Drives apps as an external black box with no app integration, so any toolkit or language works. 30 tools with text-first responses (element ids, diffs, waits) so the agent need not screenshot every step, plus optional sandboxing of the launched app. Backends: X11, Wayland, Windows, macOS, Android emulators, and the iOS Simulator.

## Claude Desktop Quick Installation
Heuristic fallback — verify the package name and runner against the repository README before running it. Uses `npx` (confidence: low):

```json
"mcpServers": {
  "glass": {
    "command": "npx",
    "args": ["-y","fixed-width-glass"]
  }
}
```

## Documentation & README

# glass

[![CI](https://github.com/fixed-width/glass/actions/workflows/ci.yml/badge.svg)](https://github.com/fixed-width/glass/actions/workflows/ci.yml)
[![glass MCP server](https://glama.ai/mcp/servers/fixed-width/glass/badges/score.svg)](https://glama.ai/mcp/servers/fixed-width/glass)

**Give your coding agent hands for the app it's building** — launch it, drive it, and verify the
result, without burning a screenshot on every step.

A Rust [MCP](https://modelcontextprotocol.io) server that gives an AI coding agent a closed **build →
see → interact → debug** loop over external native GUI applications.

glass lets an agent launch a GUI app, capture what is on screen, inject mouse and keyboard input, read
the app's logs, and detect visual changes — so a coding agent can build and debug UI applications
independently instead of asking the user "does this look right?".

glass drives apps as an external black box, so it works with any native GUI app regardless of toolkit
or language. It has two Linux backends (**X11** and **Wayland**), a **Windows** backend, an
**Android** backend (an AVD emulator, driven over `adb` from any host), an **iOS** backend (native
apps in the Simulator over `xcrun simctl`, with input and the accessibility tree via `idb_companion`,
including a two-finger pinch), and a **macOS** backend, behind a platform-agnostic core.

## See it

![An agent debugging a GTK app under glass](https://raw.githubusercontent.com/fixed-width/glass/HEAD/docs/assets/hero-debug-loop.gif)

An agent building a GUI app runs it under glass, reproduces a bug **from the accessibility tree**
(no screenshots), fixes the code, and re-verifies — the loop it otherwise can't close on its own.
[Try it yourself](#try-it-in-60-seconds).

## Try it in 60 seconds

1. Download glass for your platform from the [Releases page](https://github.com/fixed-width/glass/releases/latest)
   and [connect it to your agent](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/connect-an-agent.md) — glass works with any MCP host
   ([see which are verified](https://github.com/fixed-width/glass/blob/HEAD/docs/reference/host-compatibility.md)).
2. Get the example app — clone this repo, or download
   [`examples/tasks-demo/tasks_demo.py`](https://github.com/fixed-width/glass/blob/HEAD/examples/tasks-demo/tasks_demo.py) (on Linux it needs
   `sudo apt install python3-gi gir1.2-gtk-4.0`).
3. Paste this to your agent:

   > Use glass to run `examples/tasks-demo/tasks_demo.py` with accessibility on. There's a bug: clicking
   > **Add** doesn't add the typed task. Reproduce it by driving the UI and checking the
   > accessibility tree (don't just screenshot), then find and fix the bug in the code and verify a
   > task actually appears.

Your agent launches the app, reproduces the bug from the accessibility tree, fixes the one-line
wiring bug, and confirms the task appears — the whole build → see → interact → debug loop, start to
finish. (`glass-mcp doctor` checks your environment if anything's off.)

## The loop in practice

An agent can fill in a form, click Save, and check that the app reports success. When the app exposes
an accessibility tree, the agent works with named controls and verifies changes from text:

```jsonc
glass_start { "run": ["python3", "app.py"] }
glass_do { "actions": [
  { "action": "set_value",
    "target": { "query": "account", "role": "TextField", "states": ["enabled"] },
    "text": "hello" },
  { "action": "click_element",
    "target": { "query": "Save", "role": "Button", "states": ["enabled"] },
    "mode": "auto" },
  { "action": "wait_for_element", "name": "Saved" }
] }
glass_logs
```

Use `glass_find_elements` to inspect candidates when the intended target is not unique or known well
enough to act on directly. Use `glass_a11y_snapshot` to explore the app's controls. See the
[tool reference](https://github.com/fixed-width/glass/blob/HEAD/docs/reference/tools.md#glass_click_element) for click modes and platform behavior.

For a canvas or custom-rendered app with no accessibility tree, drive it by pixels instead —
`glass_screenshot`, `glass_click {x,y}`, and `glass_diff`, which returns `changed_pct` + a `bbox` as
text, so routine checks between screenshots cost no vision tokens. Why the loop is shaped this way:
[the build → see → interact → debug loop](https://github.com/fixed-width/glass/blob/HEAD/docs/explanation/the-loop.md).

To review a run after the app stops, opt into [session evidence recording](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/record-session-evidence.md)
with `--trace-dir`. Glass retains tool inputs and requested results in bounded storage, then
`glass-mcp trace inspect` and `trace export` validate the trace and package its evidence in a ZIP.
Recording is off by default; retained app content can contain sensitive data.

## Install at a glance

Download the latest build for your platform from the
[Releases page](https://github.com/fixed-width/glass/releases/latest), then set up your host:

- **Linux** — [docs/how-to/setup-linux.md](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/setup-linux.md) (X11 or Wayland; `Xvfb` /
  `sway` + bubblewrap)
- **Windows** — [docs/how-to/setup-windows.md](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/setup-windows.md) (a prebuilt `.exe` +
  Sandboxie)
- **macOS** — [docs/how-to/setup-macos.md](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/setup-macos.md) (install the notarized `.dmg`;
  no build needed)
- **Android** — [docs/how-to/setup-android.md](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/setup-android.md) (an AVD emulator, from any
  host)
- **iOS** — [docs/how-to/setup-ios.md](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/setup-ios.md) (the Simulator, macOS host only)

Every asset is listed in [docs/reference/platforms.md](https://github.com/fixed-width/glass/blob/HEAD/docs/reference/platforms.md#release-artifacts).
Prefer to compile, or on an architecture with no published asset? See
[docs/how-to/build-from-source.md](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/build-from-source.md) — it is a single `cargo build`.

Then [connect glass to your agent](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/connect-an-agent.md) and run `glass-mcp doctor` to check
the environment. New here? Follow [the tutorial](https://github.com/fixed-width/glass/blob/HEAD/docs/tutorial/first-drive.md) for a guaranteed first
success.

The full toolbox remains the default. For fewer agent tool definitions, start with
`glass-mcp --tool-profile lean` and use `glass_do` for actions. Inspect either inventory without
starting a session using `glass-mcp tools --json`; see [tool profiles](https://github.com/fixed-width/glass/blob/HEAD/docs/reference/tools.md#tool-profiles).

## Drive it well — the `glass-drive` skill

glass needs no app integration and no skill to run, but an agent drives it far more reliably with the
open [glass-drive](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/drive-glass-well.md) Agent Skill — it stops the agent spending its first
turns rediscovering the verify-cheaply-then-look loop. **Installing it is the single highest-leverage
thing you can add** when pointing an agent at glass.

## Platform support

**✓** supported · **◑** partial · **–** not supported · **🚧** planned.

<!-- KEEP IN SYNC with docs/reference/platforms.md (the canonical matrix) and the code. -->

| Capability | Linux (X11 + Wayland) | Windows | Android (AVD) | iOS (Simulator) | macOS |
|---|:--:|:--:|:--:|:--:|:--:|
| Capture · input · windows · clipboard · logs | ✓ | ✓ | ✓ | ✓ | ✓ |
| Accessibility (semantic addressing) | ✓ AT-SPI | ✓ UI Automation | ✓ UIAutomator | ✓ idb | ✓ AX |
| Containment / sandboxing | ✓ bubblewrap | ✓ Sandboxie | ✓ the emulator VM | ✓ the Simulator | ✓ Seatbelt |
| Display isolation (app off your desktop) | ✓ headless Xvfb / sway | ◑ virtual display · VM tier | ✓ headless emulator | ✓ headless simctl boot | 🚧 |

Full matrix, per-capability detail, and system requirements:
[docs/reference/platforms.md](https://github.com/fixed-width/glass/blob/HEAD/docs/reference/platforms.md). Transport is MCP over stdio (default) or
network HTTP.

## Documentation

The full docs — tutorial, how-to guides, reference, and explanations — are under
**[`docs/`](https://github.com/fixed-width/glass/blob/HEAD/docs/README.md)**. See [`CHANGELOG.md`](https://github.com/fixed-width/glass/blob/HEAD/CHANGELOG.md) for release notes, and
[Stability and versioning](https://github.com/fixed-width/glass/blob/HEAD/docs/reference/stability.md) for what a 1.0 release guarantees.

Contributing? [`CONTRIBUTING.md`](https://github.com/fixed-width/glass/blob/HEAD/CONTRIBUTING.md) has the gates a PR has to pass, and
[Verify a change](https://github.com/fixed-width/glass/blob/HEAD/docs/how-to/verify-a-change.md) covers platform code your own host cannot run.

## License

glass is **open core**, licensed **Apache-2.0** — see [`LICENSE-APACHE`](https://github.com/fixed-width/glass/blob/HEAD/LICENSE-APACHE).