OpenAdapt Agent

mcp-name: io.github.OpenAdaptAI/openadapt-agent

OpenAdapt compiles demonstrated GUI workflows into programs. This package invokes them over MCP.

pip install openadapt includes this package. Point a local MCP client at openadapt-agent serve --allow-run and the calling agent can invoke a compiled GUI program on this computer. Healthy runs make no model calls. Flow still runs the program. Policy, identity, verification, pauses, repair, and audit stay in openadapt-flow. --allow-run is an explicit opt-in. Admission stays fail-closed.

The calling agent operates. A named human is authority. They watch if the agent clicked, then they admit. Compliance samples Seals, and that is a different job than running the program. Computer-use agents are the user of OpenAdapt. They are not the executor inside OpenAdapt.

Don't name this skill computer use.

Install

Python 3.10 through 3.12. Local Claude Code, Cursor, Codex, and Grok CLI speak stdio MCP. Author in the chat you already use.

claude mcp add openadapt -- \
  uvx --from 'openadapt-agent[tutorial]' openadapt-agent \
  serve --allow-run

The server generates the public synthetic tutorial at serve time. It isn't in the wheel.

python -m pip install --upgrade openadapt
openadapt flow tutorial
openadapt-agent serve --allow-run

openadapt quickstart --break-it is the halt demo. After the verified run, the same certified bundle hits a backend that paints a success banner and rejects the write. An independent system-of-record read catches the lie. The record did not change.

Local unsigned replay is free. If the tool returns unsigned success, treat it as failure. Production success without a Seal is failure. HALTED does not mint a Seal.

A first demo over stdio uses a second MCP server. --authoring does not enable run tools. First-demo compile needs a human ok before the tool is callable, and that ok is the whole approval.

claude mcp add openadapt-authoring -- \
  uvx --from 'openadapt-agent[tutorial]' openadapt-agent \
  serve --authoring

--bundles is omitted. Probe tools are observe, start_record, click, and halt. This process stays stdio. Pass --url --headed to pin a fresh Playwright Chromium with empty cookies. Sign in in that window with pause_for_input; continue_input records it. --url is not the Chrome you already signed into, and it does not attach over CDP. Omit --url after you sign in in Chrome, and the unique frontmost window pins on macOS (no DOM identity). Windows native, Citrix, and RDP stay coach-only here, as does any macOS or Linux session without a unique frontmost window.

Private customer bundles still use --bundles. Those stay on the operator's disk and are never shipped here.

uvx openadapt-agent serve --bundles /path/to/bundles
uvx openadapt-agent serve --bundles /path/to/bundles --allow-run

ChatGPT.com and Claude.ai

Hosted ChatGPT.com and Claude.ai can't talk to localhost stdio. Don't tell those tabs to run openadapt-agent serve. Send them to https://openadapt.ai/start.

Those tabs still can't click the user's GUI.

Once a job exists at https://openadapt.ai/j/{id}, Desktop Connect this computer claims openadapt://runner. Copy from that page and paste into the chat.

Pip users start the mailbox client:

openadapt-agent authoring connect \
  'openadapt://runner?pack=p.…&bind=oab_…&origin=https://openadapt.ai'

That claims oab_, polls /j/{id}/runner/poll with wait_seconds: 0, prints Allow (y/n) per chat account, and on pause prints Sign in in the app, then press Enter. Continue records with record_observed. It never types your password. Credentials stay on the user's computer. Don't paste a login into a vendor VM.

The job page should offer openadapt connect <url> next to Open OpenAdapt. This repo implements openadapt-agent authoring connect. Overlay chrome, launchd, the OS URL handler, and Keychain lease persistence stay Desktop-only. See docs/MAILBOX_CLI.md.

Outcomes

Every run_workflow_<opaque-id> call returns one of success, halt, refused, timeout, or error.

Don't summarize any of them as success.

A timeout isn't a rollback. Don't retry until you've looked at the durable run.

get_run_report returns a PHI-safe status and a count-only summary for a run this server created. The persisted report stays in the local operator experience unless you enabled protected export.

Trust boundary

Don't expose this process's stdin and stdout as an unauthenticated network service. It inherits the local user's OS permissions, and Flow records that OS account as the operator for attended decisions.

By default every MCP response is safe to render outside the protected workflow-data boundary. Labels, recorded values, paths, raw reports, observed text, stdout, stderr, and local exception messages stay on the OpenAdapt machine. The client gets opaque IDs, fixed outcome copy, declared parameter names and types, and count or boolean metrics.

--allow-protected-export sends raw local metadata and evidence to the MCP client. Use it only when that client is trusted and inside the same protected data boundary.

--allow-synthetic-recorded-defaults lets omitted parameters reuse recorded values. It requires --allow-run. Synthetic demonstrations only. Production runs require every declared parameter so a wrong-record action cannot hide in a default.

Remote transport, account identity, tenant isolation, fleet policy, and managed execution belong to OpenAdapt Cloud. They aren't duplicated here.

The complete contract is in docs/DESIGN.md.

Serve a bundle

--allow-run with no --bundles records and compiles the synthetic MockMed workflow, certifies it, then keeps that app up so a governed run can hit a live system of record. --tutorial is the same path without implying run tools.

openadapt-agent serve --allow-run --runs-dir /tmp/openadapt-runs

The client gets list_workflows, get_workflow, get_run_report, list_needs_attention, and get_attention_item. Those are read-only until --allow-run. The synthetic tutorial registers run_local_quickstart. Add --allow-run and the server registers one typed run_workflow_<opaque-id> tool per loadable bundle. Declared parameters are required. Recorded demonstration values never appear in the schema and are not reused unless you ask for that.

--headed keeps the attended web session visible. The MCP server is stdio. The target app still has a window.

Finish a halt

continue_attention does not perform the paused action again. Point the server at Flow's qualified deployment config if a local operator needs to finish an exception and continue the same durable run:

openadapt-agent serve \
  --bundles /opt/openadapt/bundles \
  --runs-dir /var/lib/openadapt/runs \
  --allow-run \
  --allow-attended-actions \
  --config /etc/openadapt/deployment.yaml \
  --headed

The queue summary is safe to show in an agent UI. It carries opaque IDs, typed categories, counts, and signed-capability metadata. Observed text, workflow values, reports, and local paths stay off that surface.