MK QA Master

AI 測試大師 — your AI QA loop, from analyze to advise.

English · 繁體中文

Universal MCP server for running tests across pytest / Jest / Cypress / Go, with built-in DOM analyzer, run history, and a self-improvement coach. Stable since v1.0.0 (2026-06-02) — see Stability promise below.

A Model Context Protocol server that lets Claude Desktop / Cursor / any MCP client drive your test suite end-to-end: run tests, inspect failures (screenshot + video + trace), analyze a live URL to draft test cases, and — after each run — produce a prioritized action plan telling you exactly what to fix or write next.

Full design notes: docs/framework.md.

What's in the box

• Run tests across multiple frameworks (web + mobile + API) via a single MCP surface

• Mobile via Maestro (since v0.3.0): same MCP tools, iOS Simulator / Android Emulator / real device; YAML flows; cross-platform without rewrites

• Native API testing — two runners (since v0.6.0 / v0.6.1): two peers now share the API testing slot, each fed by the artifact your team already maintains.

  • Schemathesis (QA_RUNNER=schemathesis, since v0.6.0): point at an OpenAPI 3.x / Swagger 2.0 URL or file:// schema and get property-based fuzzed tests covering status codes, response schemas, content types, and 5xx-on-fuzz violations.
  • Newman (QA_RUNNER=newman, since v0.6.1): point at an exported Postman 2.x collection (plus optional environment / globals files) and Newman replays every request, runs the embedded pm.test(...) assertions, and returns one mk-qa-master nodeid per assertion. Newman is a system prerequisite (npm install -g newman) — it's an npm package, not pip, so it doesn't ship as a Python extra.

  Both drop into the same MCP tool surface as the web / mobile runners, and both feed the same report.json / history / flake / optimizer pipeline. Existing API tests written in pytest+httpx, Jest+supertest, Cypress cy.request(), or Go net/http/httptest still ride their existing runners — no migration needed. Pact provider verification stays on the v0.7.0 conditional roadmap.

• Failure artifacts: screenshot (base64-inlined), video, Playwright trace.zip / Maestro recordings

• Run history: every run snapshotted; HTML report shows a sparkline trend

• DOM / Screen analyzer — analyze_url for web (forms / nav / dialogs / CTAs + the API endpoints the page hits) and analyze_screen for mobile (maestro hierarchy → form / cta / tab_bar modules)

• Smart test generation (generate_test): hand it an analyzer module and it writes a runnable Playwright .py or Maestro .yaml with concrete selectors, not # TODO stubs

• Auto-retry flakes — pytest side via pytest-rerunfailures; Maestro side via custom retry wrapper (no native --reruns); flaky tests surfaced separately from real failures

• Self-improvement coach (get_optimization_plan): post-run analysis across three lenses — suite quality, MCP usability, AI generation effectiveness

• JUnit XML output for CI integrations (GitHub Actions / Jenkins / GitLab)

Install

Two paths — pick the one that matches how you'll use it.

A. Run via uvx (zero install, recommended for end users)

Add mk-qa-master to your client config without installing anything globally; uv fetches and runs it in an ephemeral environment per session:

{
  "mcpServers": {
    "mk-qa-master": {
      "command": "uvx",
      "args": ["mk-qa-master"],
      "env": { "QA_RUNNER": "pytest", "QA_PROJECT_ROOT": "/path/to/your-test-project" }
    }
  }
}

That's the whole setup. First call downloads the package; subsequent calls are cached. Switching versions: uvx mk-qa-master@0.4.1 ....

B. Install into a project venv (for contributors / hacking)

pip install mk-qa-master       # or: pip install -e . from a clone
playwright install                # only if you use pytest-playwright
pip install pytest-rerunfailures  # optional, enables auto-retry

Then point your client config at the same Python interpreter:

"command": "/path/to/.venv/bin/python",
"args": ["-m", "mk_qa_master.server"]

Verify the install (v1.4+)

mk-qa-master doctor          # human-readable check report
mk-qa-master doctor --json   # for CI gates / host-LLM consumption

Walks Python version, ffmpeg + mediamtx on PATH, core deps, [edge] extras, runner registry, and MCP tool surface. Exits 0 when nothing critical is missing (warnings about unused features don't fail), 1 when mk-qa-master can't run cleanly. Run it after a fresh install or when an MCP tool returns missing_extras.

Runner-specific prerequisites

API testing (QA_RUNNER=schemathesis)

Point the runner at any OpenAPI 3.x / Swagger 2.0 schema and Schemathesis generates property-based test cases per operation — covering response schema conformance, status code conformance, content-type checks, and 5xx-on-fuzz. Results flow through the same report.json / history / flake / optimizer pipeline as your UI tests.

End-to-end walkthrough lives in docs/walkthrough-api.md; a self-contained 3-endpoint sample lives at examples/sample_api_project/.

5-line config

"env": {
  "QA_RUNNER": "schemathesis",
  "QA_OPENAPI_URL": "https://api.example.com/openapi.json"
}

Environment variables

Standard QA_TIMEOUT_SECONDS still applies (default 600s).

API testing (QA_RUNNER=newman)

Point the runner at any exported Postman 2.x collection and Newman 6.x replays every request, runs the embedded pm.test(...) assertions, and returns one mk-qa-master "test" per assertion. Results flow through the same report.json / history / flake / optimizer pipeline as the Schemathesis and UI runners.

System prerequisite: Newman ships via npm, not pip. Install once:

npm install -g newman

There's no pip install 'mk-qa-master[postman]' extra — the runner just shells out to the newman binary on PATH. If it's missing, the runner raises a clear ImportError pointing at the npm install line.