Deskcert

What the deskcert MCP server does

The deskcert MCP server evaluates whether an AI agent can perform approved tasks in a web application without attempting actions that the suite forbids. You author the test suite for the application under review rather than using a fixed benchmark environment. This makes it suitable for internal admin panels, dashboards, CRUD tools, and other browser-accessible systems.

Each task identifies the application through a target_url, describes the work the agent should perform, and lists named forbidden_actions. The runner checks task outcomes and applies the forbidden-action policy independently from the numeric score. An attempted forbidden action is intercepted before it reaches the page, recorded with its action name and step number, and causes the suite gate to fail.

The deskcert MCP server is scoped to web applications. It does not provide native desktop automation, VM snapshots, or operating-system-level control for Windows or macOS applications.

How it works

DeskCert drives the target site with Playwright. An agent adapter receives a screenshot and an accessibility-tree text dump, then returns the next action. The project includes a scripted adapter that requires no model or API key, while the adapter interface can be connected to an external or in-house agent loop.

The MCP mode is started with deskcert mcp and exposes run_suite over standard input and output. A deployment pipeline or coordinating agent can call that tool instead of launching the command-line runner directly. The same project also provides CLI commands for initializing suites, running them, and applying CI-specific exit codes.

For CI, exit code 0 indicates a pass, 1 indicates that the score is below the configured threshold, and 2 indicates at least one forbidden-action violation. This lets a pipeline distinguish an insufficient score from an agent attempting an explicitly blocked operation. The score reflects only the tasks and guardrails defined in the suite; it is not a general certification of the agent.

Setup and configuration

Install either package before using the deskcert MCP server:

• npm: npm install -g deskcert-cli
• PyPI: pip install deskcert-cli

Install the Chromium browser used by Playwright with npx playwright install chromium for the npm installation or playwright install chromium for the Python installation.

Run deskcert init to create an example suite and fixture application, or provide your own suite directory with --suite. A suite can target a local fixture, staging system, or internal application reachable through the relevant network. The schema accepts only http:// and https:// target URLs; file:// and javascript: URLs are rejected.

The Python package also includes deskcert serve-fixture, while the npm package can run the bundled fixture server directly with Node. Both package implementations expose the same init, run, ci, and mcp command surface and are expected to produce equivalent scoring results.

Tools and capabilities

The deskcert MCP server provides the run_suite MCP tool over stdio. Its surrounding CLI supports these related capabilities:

• Create an example suite and fixture with init.
• Run a suite with a selected agent adapter through run.
• Apply CI pass, threshold, and forbidden-action gates through ci.
• Start the MCP interface with mcp.
• Intercept and record forbidden actions before page execution.
• Score task completion while keeping guardrail violations as unconditional failures.

Suites are written in YAML and validated against the included JSON Schema. The browser runner can use the bundled scripted adapter for repeatable initial tests or CI self-tests, or an implementation of the two-method AgentAdapter interface for another agent framework.

Limitations and notes

Testing is limited to browser-driven web applications. The project does not claim to assess an agent across arbitrary desktop software or a complete operating-system environment. Results depend on the tasks, success checks, forbidden actions, and threshold selected in the suite.

A successful numeric score does not override a forbidden-action violation. Conversely, the suite score is not a broad safety rating: it describes the specific application and scenarios that were executed. The MCP transport documented here is stdio, so the calling client must be able to launch and communicate with a local MCP process.