tapp — ship with proof

Tapp is the release-contract and evidence layer for teams shipping agent-authored applications. It turns a repository and real product into an observed UI Map, a compact reviewed deterministic suite, and an inspectable merge decision.

Coding agents can write the code, and (with Playwright & friends) they can even drive the app. What nobody gives them is judgment: did it actually work? tapp explores your app like a user — no test code, no app changes — detects what's broken, and commits to a verdict your merge queue can trust: ready, caution, or blocked, with evidence.

Three platforms, one judgment layer:

• iOS — the missing Playwright for iOS. tapp is hands and judgment: a generic XCUITest harness drives any app on the simulator via the accessibility surface. Native — no Appium, no WebDriverAgent.
• Android — black-box native driving through ADB + UIAutomator. Install an APK, target its application id, and run the same QA, committed Flows, evidence, and regression gate. The app does not link a Tapp SDK.
• Web (beta) — built on Playwright. Your agent already has browser hands; tapp adds the autonomous exploration, the deterministic detectors (uncaught exceptions, failed requests, dead buttons, broken links, placeholder href="#" links, error pages), and the same verdict.

you:    "Add a logout button to the settings screen"
agent:  *writes the Swift*
agent:  *tapp: builds, opens the app, navigates to Settings, screenshots it*
agent:  "Done — and here it is working on the simulator: [screenshot]"

Quickstart for coding agents

Requirements: Node ≥ 18. iOS needs macOS + Xcode; Android needs adb plus a connected emulator/device; web needs Playwright + Chromium.

From the app repository, let the agent see the current screen and then ask for a release verdict:

npx -y @aarwitz/tapp open   # builds/launches as needed; prints a screenshot path + screen summary
npx -y @aarwitz/tapp qa     # explores the real app; prints verdict, findings, and evidence report

Claude Code can read the saved image with its file-reading tool; Codex can open it with view_image. The agent should report what the screenshot proves, preserve Tapp's exact ready/caution/blocked verdict, and link the HTML evidence report. No server, account, config file, test code, API key, or bundle id is required for this loop.

The product, executable, and package leaf are all Tapp: npm distributes it as @aarwitz/tapp, while the installed command remains tapp.

To bootstrap maintained release infrastructure, preview the repository model and grounded plan before Tapp writes anything:

npx -y @aarwitz/tapp init . --dry-run --json-out /tmp/tapp-init.json
# Build/start the detected web target, ground the first UI Map, then stop it.
npx -y @aarwitz/tapp init . --explore --platform web
# Or build/install the detected Xcode target, ground the map, and persist the validated scheme.
npx -y @aarwitz/tapp init . --explore --platform ios --target .
# Or connect to an already-running owned URL:
npx -y @aarwitz/tapp init . --explore --platform web --url http://127.0.0.1:4173
# If the app has roles/accounts, bind names once; values stay in local/CI secrets.
npx -y @aarwitz/tapp actor set alice . --role member --session isolated \
  --credential email=ALICE_EMAIL --credential password=ALICE_PASSWORD
# Review-only path: tapp init . → tapp plan show → tapp plan review --approve ...

# After approved drafts replay and are promoted, establish the selected target's baseline
# through the ordinary full gate, then generate the reviewable GitHub workflow.
npx -y @aarwitz/tapp baseline create . --platform web
npx -y @aarwitz/tapp ci install .

The baseline command writes only after autonomous QA and every selected deterministic suite pass conclusively. It stores .tapp/baselines/<platform>/<target-id>.json; the generated workflow uses that exact target identity so two apps on the same platform never share a baseline. ci install writes .github/workflows/tapp.yml plus .tapp/ci.json, refuses unresolved build configuration and existing-file collisions, and never commits, pushes, enables branch protection, or creates GitHub resources. Review and pin the generated Tapp release reference to its immutable commit SHA before production.

Every verb takes whatever you have: nothing (auto-detects the repo you're in, or the app already on the simulator), a repo directory, a path/to/App.app, or a bundle id:

npx -y @aarwitz/tapp open [target]   # launch the app → screen summary + screenshot file
npx -y @aarwitz/tapp tree [target]   # accessibility tree of the current screen
npx -y @aarwitz/tapp shot            # screenshot the booted simulator
npx -y @aarwitz/tapp apps            # what's installed on the simulator (names + bundle ids)
npx -y @aarwitz/tapp build [dir]     # just build + install (scheme auto-detected)

Web (beta): npx -y @aarwitz/tapp qa http://localhost:3000 (one-time setup: npm i -g playwright && npx playwright install chromium)

Focused web inspection waits briefly for loading states to settle. If a consent or location modal blocks the screen, dismiss it and wait for the content you care about in the same package-only call:

npx -y @aarwitz/tapp open https://example.com --tap "Not now" --wait-for "Dashboard"
npx -y @aarwitz/tapp tree https://example.com --tap "Not now" --wait-for "Dashboard" --json

Android:

npx -y @aarwitz/tapp qa path/to/app-debug.apk --platform android --app-id com.acme.app
npx -y @aarwitz/tapp open com.acme.app --platform android

Optional but recommended (prebuilds the test harness so the first run is fast):

npx -y @aarwitz/tapp install    # ~2 min, one time
npx -y @aarwitz/tapp doctor     # verify Xcode / simulators / toolchain

Optional browser workspace

npx -y @aarwitz/tapp app . opens a local Release Studio for people who want visual repository onboarding, release-plan review, and CI preparation. It is not required for the coding-agent open/qa workflow.

MCP hookup (optional)

The MCP server adds the two things a CLI can't do: screenshots inline in your agent's context (the model literally sees the screen) and the interactive session loop (tap → read tree → type, with the app staying open between actions).

Claude Code:

claude mcp add tapp -- npx -y @aarwitz/tapp mcp

Cursor / VS Code (Copilot) — add to ~/.cursor/mcp.json (Cursor) or .vscode/mcp.json (VS Code):

{
  "servers": {
    "tapp": { "type": "stdio", "command": "npx", "args": ["-y", "@aarwitz/tapp", "mcp"] }
  }
}

Codex CLI — add to ~/.codex/config.toml:

[mcp_servers.tapp]
command = "npx"
args = ["-y", "@aarwitz/tapp", "mcp"]

Any other MCP client: stdio command npx -y @aarwitz/tapp mcp.

Then ask your agent:

"Run tapp qa on my app — is it ship-ready?" "Open com.mycompany.app on the simulator and screenshot the home screen." "Log in with test@example.com, drive to checkout, and record it as a replayable test."

What the agent gets

Full agent playbook: AGENTS.md — ships inside the package so agents can read it too. Application-model and import contract: docs/application-model.md. The desktop Coverage view reads the same .tapp/application-model.json, .tapp/release-plan.json, and .tapp/ui-map.json, including explicit proposal review; it does not maintain a separate product model. Map nodes identify both the real launch entry and the deterministic per-platform navigation root used for bounded changed-surface replay.

The verdict you can trust

Adaptive exploration, deterministic judgment. Exploration is adaptive — two runs may traverse different paths through your app. Judgment is deterministic: the same evidence trace always produces the same findings and verdict — no LLM variability in the decision loop. PR gating keys on the regression diff (stable finding signatures vs. a baseline), so it reacts to what changed, not to run-to-run path variance. For critical user journeys, committed Tasks and Flows provide the stable CI suite: reusable semantic actions, exact assertions, condition-based waits, fresh launch state, bounded timeouts, and evidence on failure. We call this flake-resistant, not magically flake-free—backend outages, unstable test data, and poorly identified controls can still make any E2E test fail.

Native has a heuristic release score; exploratory web does not. The native 0–100 number comes from fixed deductions and is not calibrated probability. Web reports deterministic findings, advisory budget-capped control probes, and concrete coverage instead of compressing those unlike signals into a scalar. Committed Flows, Tasks, contracts, and baseline regressions provide the web merge decision.

tapp_run_qa explores like a user — accessibility surfaces on iOS/Android and a real browser on web — and detects crashes, failed sign-ins, dead buttons, stuck loading screens, error surfaces, navigation loops, and dead ends (plus, on web: uncaught JS exceptions, failed/5xx requests, broken links and assets, and visible placeholder links with no destination). The verdict is deterministic (no LLM in the run loop) and honest:

• blocked — a release-blocking issue was found.
• caution — issues to review, or the run couldn't see enough.
• ready — genuinely explored with no detected blockers in the checks that ran. A shallow run is never ready — if the app crashed on launch or a login wall blocked exploration, you get inconclusive: true, not a false pass. Absence of findings is not a pass.

Web beta presents a ready result as AUTOMATED CHECKS COMPLETE, not “ship-ready,” and displays no scalar score. Exhaustive checks on each exercised page drive the verdict; sampled control probes remain visible findings but are advisory. The report explicitly excludes content/claim accuracy, privacy and API data minimization, brand/SEO consistency, and subjective visual credibility. Those require reviewed contracts, privacy review, or human/vision judgment; an exploratory crawl must not imply they were validated.

For a business guarantee such as “every coach is insured,” use a deterministic app-owned verifier endpoint that returns success only when the invariant holds, then require that status and the customer-visible claim in a release contract. The current DSL does not yet read arbitrary JSON response bodies or compare a cross-origin API payload directly with page copy; use a verifier or an explicit CI preflight rather than assuming autonomous QA inferred the guarantee.

Apps behind a login? Pass testEmail/testPassword (typed into the login form automatically), appLaunchArgs (e.g. ["--uitesting"] if your app supports a bypass), or explicit loginSteps for custom login UIs.

CI gate

The same engine runs as a merge gate — explore on every PR, replay committed release contracts, Flows, and multi-actor Scenarios, diff findings against the last conclusive default-branch run, fail on regressions, post a sticky PR comment, and upload screenshots, the recording, and machine-readable JSON:

# .github/workflows/tapp.yml
name: Tapp release gate
on:
  pull_request:
  push:
    branches: [main] # refreshes the automatic baseline after merges

permissions:
  actions: read
  contents: read
  pull-requests: write

concurrency:
  group: tapp-${{ github.ref }}
  cancel-in-progress: true

jobs:
  tapp:
    runs-on: macos-15
    timeout-minutes: 45
    steps:
      - uses: actions/checkout@v4
      - uses: aarwitz/tapp@main # pin to the newest release tag for production
        with:
          project: MyApp.xcodeproj # or MyApp.xcworkspace
          scheme: MyApp

On pull requests, the Action automatically reads the complete changed-file set from GitHub, retaining old and new paths for renames. It always runs critical/policy.always contracts, adds contracts related through reviewed source ownership, the UI Map, and transitive Task composition, and records skipped contracts and coverage gaps in tapp-pr-plan.json. A selected contract that does not execute blocks the merge. Set pr-selection: "false" only when intentionally running the full contract set.

Changed weakly covered surfaces are not limited to direct web URLs. Reviewed Task source ownership can compile one bounded native target (or up to five web targets) through observed UI Map edges, with condition waits and stable target evidence. Missing or failed targets make the run inconclusive; Tapp does not guess a path from a screen name.

The first successful, conclusive run on main seeds a repository-scoped Actions cache and a 90-day baseline artifact. Both are keyed by platform and stable application-model target id. Pull requests automatically restore it and fail only on new high/critical findings or broken Flows—not pre-existing debt. No baseline commit or PAT is required. If you prefer a reviewed, durable baseline, run tapp baseline create and commit the generated .tapp/baselines/<platform>/<target-id>.json; tapp ci install wires its explicit path into the corresponding job. The legacy .tapp/baseline.json is still recognized. Automatic baseline restore and the PR comment need actions: read and pull-requests: write as shown above. Secrets are unavailable to workflows from forks, so auth-gated apps should either use a non-secret UI-testing launch argument or skip the gate for untrusted forks.

Already build the simulator app in another job, or use another CI provider? The portable command accepts that .app, detects its bundle id, writes report artifacts, and exits non-zero when the gate fails:

npx -y @aarwitz/tapp ci --app path/to/MyApp.app \
  --project-dir . --pr-base origin/main --pr-head HEAD \
  --target-key target_ios_myapp \
  --pr-plan-out tapp-pr-plan.json \
  --baseline path/to/last-main-report.json \
  --json-out tapp-report.json --md-out tapp-report.md

See the self-test at .github/workflows/tapp-gate-example.yml for Flows, auth inputs, and other controls. GitHub-hosted iOS runs require a macOS runner; the first run also builds the XCUITest harness, so budget roughly 5–10 minutes depending on app size.

Android CI runs on Linux with an emulator/device already connected. The Action can build the APK or accept a prebuilt one:

- uses: aarwitz/tapp@main
  with:
    platform: android
    android-app-id: com.acme.app
    android-project: android
    android-task: :app:assembleDebug
    flows: android/.tapp/flows/*.yml

For web, pass platform: web plus web-target: and Tapp uses the application model to run its lockfile-backed install/build, start a detected package script or read-only static server, wait for readiness, gate it, and stop it even on failure. Pass url: instead for an already-running owned environment. Add scenarios: .tapp/scenarios/*.yml to gate isolated cross-account journeys; see docs/scenarios.md. Automatic baselines are isolated by platform and target, so two same-platform apps are never compared.

The hosted service is under development and is not currently offered for customer repositories. Do not upload private code or credentials to an old preview. The retained cloud prototype is not the production SaaS boundary. Use the local Release Studio and the portable GitHub Action in infrastructure you control until the new account, tenant authorization, private evidence, and isolated-worker boundary passes security review.

Make your repo agent-verified

Drop this into your repo's AGENTS.md (read by Codex, Cursor, Copilot, Devin, Zed, …) so your agent proves its UI work instead of claiming it:

## Verifying UI changes
This repo uses tapp (https://github.com/aarwitz/tapp) to verify UI work on a real app surface
(iOS simulator, Android emulator/device, or a browser for web). After any UI change, run `npx -y @aarwitz/tapp open` from the
repo root (it finds and builds the Xcode project itself) and look at the screenshot it saves as
proof. Before declaring a feature done, run `npx -y @aarwitz/tapp qa` (or `qa <url>` for web) and
report the ship/no-ship verdict. A change is not "done" until it has been seen working.
(If the tapp MCP server is connected, the tapp_* tools do the same with inline screenshots —
tapp_build builds + installs the app and returns the bundle id for tapp_run_qa.)

How it works

Every driver speaks one protocol: structured OCQA_* markers (state, actions, issues, transitions) that the judgment layer parses into trees, screenshots, findings, and the verdict. On iOS, a generic XCUITest harness attaches to any app by bundle id — no SDK or app code changes — and acts through the accessibility tree. On Android, ADB + UIAutomator provide the same black-box driver contract. On web (beta), a deterministic Playwright crawler does the same in a real browser. Same detectors' spirit, same dedup, same regression gate, same honest verdict. Core exploration, evidence collection, and verdict calculation run entirely locally — no telemetry, nothing phones home. Optional AI features are explicit: finding enrichment requires TAPP_ENABLE_REMOTE_AI=1 (an ambient API key alone never changes data handling), and AI flow generation / assert_ai only run when you invoke them; these send selected metadata (screen names, finding titles) to your configured model provider. Runtime configuration uses TAPP_* environment variables. Repository artifacts live under .tapp/, and desktop run configuration uses .tapp.yml. Existing .autotap/, .autotap.yml, and AUTOTAP_* inputs remain readable as migration fallbacks; when both names exist, Tapp uses the canonical one. Rename the old directory/config in your next reviewed repository change—no secrets or evidence formats need to be rewritten.

Committed Flow replay, recording a driven session, autonomous exploration, exact assertions, regression comparison, and CI gating require no API key and no coding agent at runtime. AI is only an optional authoring/enrichment layer (tapp_flow_generate, assert_ai, finding enrichment).

The first tool call builds the harness once (~2 min, cached in ~/.tapp; rebuilt automatically if you switch simulators). All captures land in ~/.tapp/captures/.

Desktop status

The macOS cockpit is frozen as a supported native interface and parity floor; it still reads the canonical Application Model, release plan, and UI Map in Coverage. Its older import/build path is not yet a thin client of the shared product-operation layer, so new product work is converging in the browser without deleting or reducing the desktop experience.

License

MIT