qa-probe

Find out exactly why your pages are blank - in under 5 minutes.

qa-probe is a Node.js command-line tool that reads your React frontend, maps every API call to its backend route, probes those endpoints live with real auth, and tells you the root cause when something is wrong - not just a status code.

Overall score: 74/100

Root causes detected:
  empty_db:              5 route(s)   seed the database
  feature_flag_disabled: 3 route(s)   enable HAS_BILLING, HAS_REPORTS
  contract_mismatch:     1 route(s)   frontend calls /api/users/, backend serves /users

Route          Score   Status     Root cause
-----------    -----   --------   ----------------------
/dashboard     100     healthy
/users          80     healthy    empty_db
/billing         0     broken     feature_flag_disabled
/reports        50     degraded   empty_db
/settings       85     healthy

It works best on React apps backed by FastAPI, Express, Next.js, GraphQL, or tRPC with an OpenAPI spec available. Built by the LightShield SIEM team and battle-tested against a production SIEM.

The problem it solves

Your dashboard loads. Nothing crashes. But every table is empty, every chart shows zero, and you have no idea why. The cause is almost always one of:

• The frontend calls /api/users/ but the backend route is /api/users (trailing slash)
• A feature flag (HAS_BILLING=false) silently disabled an entire router
• The database is connected but the table has no rows
• A backend refactor renamed a field and the frontend still reads the old name
• An endpoint is missing its auth guard, or leaks data without a token

qa-probe builds a source-aware map from frontend calls to live backend responses, then explains each failure in one line with a fix hint. It complements Playwright (user journeys) and Schemathesis (contract fuzzing) - it is the layer that answers "why is this page blank?" first.

Quick start

1. Install

npx qa-probe run        # use without installing
# or
npm install -g qa-probe

2. Create a config

qa-probe.config.js in your project root:

module.exports = {
  baseUrl: 'http://localhost:8000',      // your backend
  frontendSrc: './frontend/src',          // your React source
  routerFile: './frontend/src/App.tsx',   // your router file

  auth: {
    type: 'bearer',
    loginUrl: '/auth/login',
    credentials: {
      username: process.env.QA_USER,       // never hardcode credentials
      password: process.env.QA_PASS,
    },
    tokenPath: 'access_token',
  },
};

3. Run it

QA_USER=tester QA_PASS=secret npx qa-probe run

qa-probe will parse your frontend, fetch /openapi.json, probe every endpoint with real auth, and write a full report to .qaprobe/ (report.md, report.json, ai-context.md, report.html). A summary table prints to the terminal.

Understanding the report

Route status

Common root causes

Every result is verifiable. Each diagnosis carries its evidence (the request, a snapshot of the response, timing) and a confidence level - high, medium, or none. A confidence: none / unknown result is explicitly not a confirmed pass.

Score: each route is 0-100; the overall score is the average. Use --fail-under 80 to make CI fail when the score drops.

Use cases

• "Why is this page blank?" - the everyday case. Get a root cause and a fix hint in one run instead of digging through network tabs and logs.
• CI gate - qa-probe run --fail-under 80 blocks a deploy when pages regress.
• Demo / staging sign-off - confirm an environment actually works before a customer call, not just that it returns 200s.
• Catch contract drift - a backend refactor renames a field or moves a route; qa-probe flags the exact endpoint and frontend call affected.
• Security smoke - find endpoints missing an auth guard or leaking PII (opt-in, read-only, see Security checks).
• Logic checks - assert response invariants (valid enum values, non-negative counts, required fields) without writing a test suite.
• AI-assisted debugging - let Claude, Cursor, or Codex query the QA state and explain failures in plain English (see below).

Use it with an AI assistant (Claude Code, Cursor, Codex)

qa-probe ships a Model Context Protocol (MCP) server. Point your AI assistant at it and the assistant can read your QA data and explain failures directly - no manual report-reading.

Setup

Run qa-probe once to populate the cache:

npx qa-probe run

Then add the MCP server. For Claude Code / Cursor, add to .mcp.json at your repo root:

{
  "mcpServers": {
    "qa-probe": {
      "command": "npx",
      "args": ["qa-probe", "mcp"]
    }
  }
}

Restart the assistant. Now you can ask in plain English:

• "Why is /reports showing no data?"
• "Which pages break if the users endpoint goes down?"
• "Show me all broken routes."
• "Mark the empty /alerts result as expected." (records feedback, reapplied next run)

Available MCP tools

MCP output is sanitized - SQL errors, stack traces, and table names are redacted before they reach the assistant. Raw data stays on disk. Results carry their evidence and confidence, and explain_failure returns a plain-English trust note when a finding is unverified, so the assistant does not report guesses as passes.

You can also simply open the qa-probe folder (or your project with .qaprobe/) in an AI coding tool and ask it to read ai-context.md - a compact, LLM-oriented summary written on every run.

CLI commands

qa-probe run       [--config <path>] [--fail-under <N>]   analyze + probe + report
qa-probe analyze   [--config <path>]                       build the dependency graph
qa-probe probe     [--config <path>]                       probe all endpoints
qa-probe report    [--config <path>] [--fail-under <N>]   score, classify, write reports
qa-probe label <endpoint> <verdict> [-r <reason>]          record feedback (reapplied later)
qa-probe fix       [--config <path>] [--apply] [--pr]      generate remediation diffs
qa-probe mcp       [--config <path>]                       start the MCP server (stdio)

--fail-under <N> exits non-zero when the overall score is below N (a CI gate).

Capabilities