mirroir-mcp

Give your AI eyes, hands, and a real iPhone. An MCP server that lets any AI agent see the screen, tap what it needs, and figure the rest out — through macOS iPhone Mirroring. Experimental support for macOS windows. 33 tools, any MCP client.

Requirements

• macOS 15+
• iPhone connected via iPhone Mirroring

Install

/bin/bash -c "$(curl -fsSL https://mirroir.dev/get-mirroir.sh)"

or via npx:

npx -y mirroir-mcp install

or via Homebrew:

brew tap jfarcand/tap && brew install mirroir-mcp

The first time you take a screenshot, macOS will prompt for Screen Recording and Accessibility permissions. Grant both.

claude mcp add --transport stdio mirroir -- npx -y mirroir-mcp

{
  "servers": {
    "mirroir": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "mirroir-mcp"]
    }
  }
}

{
  "mcpServers": {
    "mirroir": {
      "command": "npx",
      "args": ["-y", "mirroir-mcp"]
    }
  }
}

codex mcp add mirroir -- npx -y mirroir-mcp

[mcp_servers.mirroir]
command = "npx"
args = ["-y", "mirroir-mcp"]

git clone https://github.com/jfarcand/mirroir-mcp.git
cd mirroir-mcp
./mirroir.sh

How it works

Every interaction follows the same loop: observe, reason, act. describe_screen gives the AI every text element with tap coordinates (eyes). The LLM decides what to do next (brain). tap, type_text, swipe execute the action (hands) — then it loops back to observe. No scripts, no coordinates, just intent.

Beyond the basics, the server exposes higher-level navigation and lifecycle tools: press_back navigates back by OCR-tapping the "<" chevron (with a canonical-position fallback), press_home/press_app_switcher/spotlight drive system gestures, scroll_to scrolls until a target text becomes visible, reset_app force-quits an app via the App Switcher, set_network toggles connectivity through Settings, and measure times how long an action takes to surface a target element. Multi-target setups use list_targets/switch_target to move between window automation endpoints. Skill authoring is served by record_step and save_compiled (compiled replay) and calibrate_component (test a component definition against the live screen). See the Tools Reference for all 33 tools.

Describe Your App

mirroir can explore any iOS app blindly, but it works better when you tell it what to expect. Write an APP.md file and mirroir reads it before exploration starts:

---
app: Santé
archetype: dashboard
obstacle_mode: auto
---

## Structure
Dashboard with 4 tabs: Résumé, Partage, Parcourir, Profil.

## Résumé Tab
- Summary cards for health metrics that drill down to charts
- Cards often show "Aucune donnée" on test devices

## Obstacles
- Health Access permission → tap "Autoriser"
- Notification permission → tap "Ne pas autoriser"

## Skip
- Supprimer les données de Santé
- Réinitialiser

What the code actually uses today: archetype overrides recipe auto-detection; obstacles are auto-dismissed when obstacle_mode: auto; skip merges with permissions.json.skipElements; tabs (inline or as a section) are injected as high-priority targets; Structure + tab body + Tips become AI context in generated skills.

See the APP.md specification for the complete field list, loader resolution rules, and the permission-system bridge. Three levels of patterns work together — elements (what rows look like), screens (what the page layout means), and apps (what the developer knows). Patterns & Skills covers the full system.

Examples

Paste any of these into Claude Code, Claude Desktop, ChatGPT, Cursor, or any MCP client:

Open Messages, find my conversation with Alice, and send "running 10 min late".

Open Calendar, create a new event called "Dentist" next Tuesday at 2pm.

Open my Expo Go app, tap "LoginDemo", test the login screen with
test@example.com / password123. Screenshot after each step.

Start recording, open Settings, scroll to General > About, stop recording.

Screen Intelligence

describe_screen is the AI's eyes. Three backends work together to give the agent a complete picture of what's on screen — text, icons, and semantic UI structure.

Apple Vision OCR (default)

The default backend uses Apple's Vision framework to detect every text element on screen and return exact tap coordinates. This is fast, local, and requires no API keys or external services.

Icon Detection (YOLO CoreML)

Text-only OCR misses non-text UI elements — buttons, toggles, tab bar icons, activity rings. Drop a YOLO CoreML model (.mlmodelc) in ~/.mirroir-mcp/models/ and the server auto-detects it at startup, merging icon detection results with OCR text. The AI gets tap targets for elements that text-only OCR cannot see.

AI Vision Mode (embacle)

Instead of local OCR, describe_screen can send the screenshot to an AI vision model that identifies UI elements semantically — cards, tabs, buttons, icons, navigation structure — not just raw text. This produces richer context for the agent, especially on screens with complex layouts.

The embacle runtime is embedded directly into the mirroir-mcp binary via Rust FFI. describe_screen calls the embedded runtime in-process — no separate server, no network round-trip, no additional setup. The FFI layer (EmbacleFFI.swift → libembacle.a) handles initialization, chat completion requests, and memory management across the Swift/Rust boundary.

embacle routes vision requests through already-authenticated CLI tools (GitHub Copilot, Claude Code) so there is no separate API key to manage. If you have a Copilot or Claude Code subscription, you already have access.

Install

brew tap dravr-ai/tap
brew trust --tap dravr-ai/tap # recent Homebrew refuses to load untrusted taps
brew install embacle          # CLI tools (embacle-server, embacle-mcp)
brew install embacle-ffi      # Rust FFI static library (libembacle.a)

Both are optional. Without embacle-ffi the build links against Apple Vision and describe_screen uses local OCR.

Then rebuild mirroir-mcp from source (or reinstall via Homebrew) so the binary links against libembacle.a:

# From source
swift build -c release

# Or via Homebrew (rebuilds automatically)
brew reinstall mirroir-mcp

Zero-config activation

When the embacle FFI is linked into the binary, screenDescriberMode defaults to "auto" which automatically resolves to vision mode. No settings change required — install embacle-ffi, rebuild, and describe_screen starts using AI vision.

To force local OCR even when embacle is available, explicitly set "ocr":

// .mirroir-mcp/settings.json
{
  "screenDescriberMode": "ocr"
}

See Configuration for all available settings.

Skills

When you find yourself repeating the same agent workflow, capture it as a skill. Skills are SKILL.md files — numbered steps the AI follows, adapting to layout changes and unexpected dialogs. Steps like Tap "Email" use OCR — no hardcoded coordinates.

Place files in ~/.mirroir-mcp/skills/ (global) or <cwd>/.mirroir-mcp/skills/ (project-local).

APP.md

Describe your app's structure to guide exploration — see Describe Your App above and the full APP.md specification. Place APP.md files in ~/.mirroir-mcp/skills/ or the mirroir-skills repo at patterns/apps/.