AI Agent Readiness for Magento 2

ExpandingWeb_AgentReadiness runs the eight public agent-readiness checks against your own Magento 2 / Adobe Commerce store from inside the admin, and counts the AI agent traffic (GPTBot, ClaudeBot, PerplexityBot, …) your storefront already gets.

It is the free, open-source tier of the Agentic Commerce Suite by Expanding Web. MIT licence. Nothing leaves your store by default (see Privacy).

The eight checks

Same check ids, semantics and verdict language as the public scanner at https://expandingweb.com/#scan — a score means the same thing in your admin, on the public scan and in the quarterly benchmark.

Reading the result:

• Score is checks passed out of applicable checks. It is a readiness measure, not a ranking prediction — nobody can promise placement in an AI answer, and this module does not.
• N/A appears when a check cannot run (no product catalog to sample, the edge refused that one path, the 60-second budget ran out). It does not count against the score.
• Blocked (the scan stops before the checks) means the store refused the scanner's own user agent at the edge; unreachable means nothing answered. Neither is reported as eight failed checks, but every real agent meets the same wall, so an edge rule is usually the first thing to fix.

Where the numbers come from: in the Q3 2026 benchmark of 1,867 Magento stores, 33.9% passed none of the eight checks; on SCHEMA-01 alone, 54% of reachable product pages carried no Product+Offer JSON-LD.

Installation

composer require expandingwebcom/module-agent-readiness
bin/magento module:enable ExpandingWeb_AgentReadiness
bin/magento setup:upgrade
bin/magento cache:flush

Requirements: Magento Open Source / Adobe Commerce 2.4.6 or newer, PHP 8.2 or newer, guzzlehttp/guzzle 7 (already part of Magento). Two tables are created: expw_readiness_scan and expw_readiness_daily. bin/magento module:uninstall ExpandingWeb_AgentReadiness drops both and every expw_readiness/* config value.

The module can be installed alongside the Agentic Commerce Suite — tables, config paths, routes, ACL ids and cron jobs do not collide.

In the admin

• Marketing → AI Agent Readiness → Readiness Dashboard: the latest scan as a score gauge and one row per check (short verdict, severity, and the scanner's full technical verdict folded under Technical details), a Run scan now button, and the agent-traffic cards.
• Stores → Configuration → ExpandingWeb → AI Agent Readiness: counting on/off, retention, daily cron scan on/off, TLS verification.

The scan runs on your own server against the default store view's base URL, from the outside in — the same fetches an agent would make: /, /robots.txt, /.well-known/ucp, /llms.txt, /sitemap.xml, up to three product pages, the homepage again as GPTBot and as ClaudeBot, and an MCP handshake if a profile advertises an endpoint. Budget: 60 seconds wall clock, 5 seconds per request, 2 MB per response. Cron expw_readiness_scan runs it daily at 03:15 server time; the last 400 scans are kept.

Run scan now is synchronous: the admin POST waits for the scan, so PHP-FPM's request timeout needs to clear 60 seconds.

How the agent-traffic counter works — and what it cannot see

An observer on controller_action_predispatch matches the request's User-Agent against a list of known AI agent tokens (Model/Tracking/AgentDetector: OAI-SearchBot, ChatGPT-User, GPTBot, Claude-User, Claude-SearchBot, ClaudeBot, Perplexity-User, PerplexityBot, Google-Extended, Applebot-Extended, Amazonbot, Meta-ExternalAgent, Bytespider, CCBot, Bingbot) and, on a match, upserts one row per (day, agent, store view) in expw_readiness_daily. Nothing happens for the other 99.x% of requests beyond one substring scan. "Day" is the store's configured timezone. Rows older than the retention setting (default 400 days) are pruned nightly. The token list is compiled into the module and updated with a module release — a new agent means a new version, never a runtime download.

The dashboard shows 7/30/90-day totals per agent, the change against the previous 30 days, and a 14-day sparkline.

The count is a floor, not a total, for two reasons the dashboard states next to the number:

1. Attribution is by declared user-agent. An agent that presents a browser user-agent is invisible to it.
2. The observer runs in PHP. A page served entirely from Varnish or the built-in full-page cache never reaches PHP, so cache hits by agents are not counted. On a store with a high hit rate the counter sees mostly cache misses — first visits, uncached URLs, and everything that bypasses the cache.

For a complete picture, count at the edge (CDN or web-server logs). This module deliberately does not ship an edge-log parser.

Privacy

The module makes no request to anything but your own store:

• The self-scan fetches your own storefront URLs (listed above), with the User-Agent ExpandingWebAgentReadiness/1.0.0 (+https://expandingweb.com/; readiness self-scan). The one exception is MCP-01: if your UCP profile advertises an MCP endpoint on another host, the handshake goes there — because that is what the profile tells agents to do.
• The agent-traffic counter reads the request's User-Agent header and writes an integer to your own database. No IP addresses, no URLs, no headers are stored.
• Scan results and counters live in your database and are shown only in your admin. The dashboard's link to the Agentic Commerce Suite is a static URL with no parameters — it carries nothing about your store.

No telemetry, no licence check, no agent-list download, no phone-home.

Keeping the checks in lock-step with the public scanner

Model/Scanner/Scanner.php and Model/Scanner/UcpProfile.php are generated, not hand-maintained: bin/port-scanner.php re-ports the public scanner's source (app/Agentic/Scanner.php in the expandingweb.com repo, which is not public) into this module's namespace and records the upstream revision in Model/Scanner/.upstream-hash. The script applies exactly five deliberate differences — namespace and strict types, ScanHttp in place of the public scanner's SSRF-hardened client (the only target here is your own base URL, which may legitimately be a private address), sequential instead of concurrent phase-1 fetches, a 60-second instead of 15-second budget (nobody waits on a cron), and the scanner's own identity in the MCP clientInfo and robots.txt matching — and refuses to run if upstream changed shape.

The reason is the promise on the tin: the public scan, the benchmark and the in-store scan have to give the same verdict for the same store. Hand-merging 1,800 lines twice a quarter is how that promise would quietly die, so the file carries a DO NOT EDIT BY HAND marker that CI enforces. Found a check misbehaving? Open an issue with the store shape that trips it; the fix lands upstream and the port is re-run. See CONTRIBUTING.md.

What is not in this module

Merchant-feed syndication (OpenAI, Perplexity, Google Merchant Center), Product/Offer/ProductGroup JSON-LD repair, the agent endpoints (/llms.txt, /.well-known/ucp, MCP server, agent product feed, checkout sessions), agent funnel and revenue-exposure analytics, AI visibility and share-of-voice probes and competitor analysis are the paid tier: the Agentic Commerce Suite. This module diagnoses; the Suite repairs.

For agents and MCP clients