# AI Visibility Checker [Health: Active] **Category:** ๐Ÿ›’ E-Commerce **Repository:** https://github.com/krisdiallo/ecom-agent **GitHub Stars:** 0 **Views:** 0 **Installs:** 0 **Upvotes:** 0 **Directory Page:** https://allmcps.com/mcp/ai-visibility-checker ## Description Check whether AI assistants can read and buy from your ecommerce store. Read-only. ## Claude Desktop Quick Installation Install path detected from listing signals. Uses `uvx` (confidence: high): ```json "mcpServers": { "ai-visibility-checker": { "command": "uvx", "args": ["--from","git+https://github.com/krisdiallo/ecom-agent","aivis-mcp"] } } ``` ## Documentation & README # aivis โ€” can AI assistants actually read your store? **One command. No install, no dependencies, no account, nothing uploaded.** ```bash curl -sO https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/aivis.py python3 aivis.py yourstore.com ``` ``` aivis 1.0.0 โ€” can AI assistants read brooklinen.com? 1. robots.txt โ€” are you blocking the crawlers that recommend you? [ OK ] 4 search crawler(s) can read your catalog OAI-SearchBot PerplexityBot Claude-SearchBot Claude-User 2. Product page โ€” can a crawler read your facts? [FAIL] Structured data is injected by JavaScript โ€” crawlers never see it [FAIL] Your contradicts your own og:title title: Classic Cotton Sheet Set | Brooklinen og:title: Super-Plush 4-Piece Bath Towel Set [WARN] Only 0 concrete measurement(s) in the readable text Summary: 2 critical 2 to review 1 passed ``` That output is real, from a live page on a major DTC brand, reproduced across three independent fetches. Prefer a browser? Same checks, paste-based: **[the web version](https://krisdiallo.github.io/ecom-agent/ai-visibility.html)**. One page is an anecdote. Sample your catalogue to see whether a problem is systemic: ```bash python3 aivis.py yourstore.com --pages 10 ``` ``` [FAIL] Structured data injected by JavaScript (7/10 pages) [WARN] Fewer than 5 concrete measurements (9/10 pages) [ OK ] 1/10 pages clean on every check ``` --- ## What this is, in the words you'd search for it This is a free **GEO tool** โ€” **generative engine optimization**, also called **LLM SEO** or **answer engine optimization**: making sure an AI assistant can read your store well enough to cite or recommend it. It is the ecommerce-specific version of the broader **AI search optimization** question, focused on the two things that decide whether an answer engine can use your content at all: 1. **Your robots.txt** โ€” are you disallowing the crawlers that feed answers (`OAI-SearchBot`, `PerplexityBot`, `Claude-SearchBot`)? Most "block AI" advice gets this wrong by conflating them with training crawlers. 2. **Your product page's structured data and facts in raw HTML** โ€” what a crawler that does not run JavaScript actually receives, not what dev tools show you. If you searched for an **AI visibility checker** or a **robots.txt checker** for an ecommerce store, this does both, with primary-source data on 165 AI crawlers rather than generic advice. --- ## Or give it to your AI An MCP server, so Claude Code, Claude Desktop, Cursor or any MCP client can run the check itself โ€” "is my store readable by AI assistants?" answered in the conversation you're already in. Nothing to download, no paths to edit โ€” paste this into your MCP config: ```json { "mcpServers": { "aivis": { "command": "uvx", "args": ["--from", "git+https://github.com/krisdiallo/ecom-agent", "aivis-mcp"] } } } ``` Also listed in the **[official MCP registry](https://registry.modelcontextprotocol.io)** as `io.github.krisdiallo/aivis`, so any client that reads the registry can find it, and there is a one-file `aivis_mcp.py` in this repo if you would rather run it directly. Stdio JSON-RPC, **zero dependencies** โ€” the whole tool is standard library, which is why it can be fetched and run straight from git. It imports `aivis.py` rather than shelling out, so the CLI, web tool, study, GitHub Action and MCP server all share one verified implementation. ## Or run it in CI Catch it on the deploy that breaks it, not months later when you wonder why an assistant never mentions you: ```yaml - uses: krisdiallo/ecom-agent@v1 with: site: yourstore.com fail-on: critical # critical | warning | never ``` Writes a findings table to the job summary, exposes `critical` / `warnings` / `passed` / `report` as step outputs, and fails the build on real problems. This repo [runs it against its own site](https://github.com/krisdiallo/ecom-agent/blob/HEAD/.github/workflows/self-check.yml) on every push โ€” a tool that tells you to check your AI visibility should be checking its own, in public. `--json` works standalone too, if you'd rather wire it into something else: ```bash python3 aivis.py yourstore.com --json --fail-on warning ``` ## The one thing most advice gets backwards There are two completely different kinds of AI crawler, and blocking them has **opposite** consequences: | Kind | Tokens | If you block it | |---|---|---| | **Search / answer** | `OAI-SearchBot` `PerplexityBot` `Claude-SearchBot` `Claude-User` `Amzn-SearchBot` `Amzn-User` `Applebot` | **You disappear from AI answers.** | | **Training** | `GPTBot` `ClaudeBot` `Google-Extended` `CCBot` `Amazonbot` `Applebot-Extended` `meta-externalagent` | Nothing changes in recommendations. A legitimate choice. | **Blocking `GPTBot` does not remove you from ChatGPT's recommendations.** `OAI-SearchBot` is the token that does. The "block AI scrapers" wave conflated these, and a lot of sites opted out of training thinking they were protecting something else. Three more that circulating robots.txt snippets routinely get wrong: - **`Amazonbot` vs `Amzn-SearchBot`.** Amazon splits the roles. `Amazonbot` "may be used to train Amazon AI models"; `Amzn-SearchBot` is what makes you "eligible to appear in search experiences such as Alexa" and "does not crawl content for generative AI model training." Most guides name only the first. Block the wrong one and you lose Alexa for nothing. - **`Applebot-Extended` does not crawl anything.** It is purely an opt-out signal. Apple states pages that disallow it "can still be included in search results." - **`facebookexternalhit` is not an AI crawler.** It renders your link previews. It gets swept into "block AI bots" lists, which then silently breaks how your links look when shared. ## Blocking AI is two decisions, not one `ai-robots-txt/ai.robots.txt` (~4k stars) is the ecosystem's default answer to "block AI bots". Its generated `robots.txt` disallows all 166 known AI user-agents. Audited with this tool: ``` $ python3 -c "import aivis,urllib.request as u; \ print(aivis.audit_robots(u.urlopen('https://raw.githubusercontent.com/\ ai-robots-txt/ai.robots.txt/main/robots.txt').read().decode()))" visible_to_ai_search: False blocked_search : OAI-SearchBot, PerplexityBot, Claude-SearchBot, Claude-User, Amzn-SearchBot, Amzn-User, Applebot ``` **That is deliberate on their part, and correct for their audience.** Their FAQ states the position plainly: these crawlers are *"extractive, confer no benefit to the creators of data they're ingesting."* A publisher who does not want ChatGPT answering questions from their articles without a click wants exactly this file. We are not describing a bug, and an earlier version of this section wrongly implied one. The point is that **blocking AI is two decisions, not one**, and a single list collapses them: - *"Don't train on me."* Costs you nothing in recommendations. - *"Don't answer questions using me."* Removes you from ChatGPT search, Perplexity, Claude, Alexa, Siri and Spotlight. A publisher usually wants both. **An ecommerce store usually wants the first and not the second** โ€” being absent from AI answers is lost demand, not protected work. If that is you, this generates the other file: ```bash python3 aivis.py --training-optout >> robots.txt ``` Blocks all 8 training crawlers, leaves all 7 search crawlers allowed. Verified in CI by auditing our own generated output. ## Use it as a library If you're building a GEO tool, a crawler, or a shopping agent, you probably want the primitive rather than the report: ```python from aivis import classify_crawler, audit_robots classify_crawler("GPTBot")["blocking_effect"] # 'opts_out_of_training_only' classify_crawler("OAI-SearchBot")["blocking_effect"] # 'removes_from_ai_answers' classify_crawler("SomeRandomBot") # None โ€” it won't guess r = audit_robots(open("robots.txt").read()) r["visible_to_ai_search"] # False if any AI *search* crawler is blocked r["blocked_search"] # the ones that actually cost you answers r["blocked_training"] # blocking these costs nothing; listed separately on purpose ``` `audit_robots` applies real group precedence โ€” a crawler obeys its own group and ignores `User-agent: *` when it has one โ€” and classifies paths so a stock Shopify file, with its ~45 default `Disallow` rules, doesn't read as broken. Every classification is sourced to the vendor's own documentation ([`crawlers.json`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/crawlers.json) carries the quotes and dates), and [`research/test_api.py`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/research/test_api.py) is a contract test in CI so the values can't drift. Zero dependencies: `pip install git+https://github.com/krisdiallo/ecom-agent`. ## `agent-commerce.json` โ€” which stores an AI agent can actually buy from **[`agent-commerce.json`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/agent-commerce.json)** โ€” 70 storefronts probed, **49 expose a live endpoint** an agent can call to search a catalogue, build a cart and complete a checkout, bypassing HTML entirely. The result worth the file: **all 49 exposed an identical 13-tool surface on one UCP version. Zero variation across 49 independent brands.** That means agent-commerce capability is currently a property of the **platform**, not a merchant choice. A store has it because its platform switched it on. Two consequences: - a merchant cannot meaningfully "optimise" this axis today beyond choosing a platform โ€” which is the opposite of how this is usually sold - an agent developer can assume a uniform tool surface across these stores rather than negotiating capabilities per merchant ```bash # every store in the sample an agent can transact with curl -s https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/agent-commerce.json \ | python3 -c "import json,sys;[print(h['host']) for h in json.load(sys.stdin)['hosts'] if h['agent_commerce']]" ``` **A live endpoint does not mean any agent can buy.** Probing further: `tools/list` answers anonymously, but every tool *call* is refused without a published UCP agent profile URI (`invalid_profile_url`), verified on three independent stores. So these stores are transactable by **identified** agents, not anonymous ones โ€” capability enumeration is open, transacting is not. That is an accountability layer worth knowing about, and our first phrasing was too strong. We did not create an agent profile to test past it. Manufacturing an identity in order to transact on someone else's storefront is not something we will do for a cleaner dataset. Probe was read-only throughout: `tools/list` only. It never created a cart or started a checkout. ## `crawlers.json` โ€” the data behind all of this **[`crawlers.json`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/crawlers.json)** is the machine-readable registry the tools are built on: 21 tokens, each with its vendor, purpose, what blocking it actually does, whether it honours robots.txt, **the vendor's own words, a source URL, and the date it was checked.** ```bash # every token whose blocking removes you from AI answers curl -s https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/crawlers.json \ | python3 -c "import json,sys;[print(c['token']) for c in json.load(sys.stdin)['crawlers'] \ if c['blocking_effect']=='removes_from_ai_answers']" ``` Three entries (`CCBot`, `Bytespider`, `Amazonbot`'s siblings aside) could not be tied to a first-party quote; those carry an explicit `verification` flag rather than being dressed up as sourced. `research/test_consistency.py` fails the build if the CLI and the registry ever disagree โ€” which is how the Amazon misclassification above was caught before publication. MIT. Use it in your own tool; a link back is welcome but not required. ## `crawler-consequences.json` โ€” the whole ecosystem, and where the data runs out `crawlers.json` is deep but narrow: 21 tokens, each with a vendor quote. The ecosystem's list of *which* AI user-agents exist, [`ai-robots-txt/ai.robots.txt`](https://github.com/ai-robots-txt/ai.robots.txt) (MIT), is the reverse โ€” 166 tokens, but its `function` field answers "what is this bot" rather than "what does blocking it cost me". Their FAQ invites reuse: *"Can I use `robots.json` directly in my own tooling? You're welcome to."* **[`crawler-consequences.json`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/crawler-consequences.json)** joins them: 165 tokens, each carrying a `blocking_effect` and a `basis` saying how strongly that is known. (165 rather than 166 because upstream lists three crawlers under two spellings each โ€” `Meta-ExternalAgent` and `meta-externalagent`. robots.txt matches user-agents case-insensitively, so those are one crawler, and emitting both would inflate any count a consumer derives from the file.) | `basis` | n | What it rests on | |---|---|---| | `vendor-documented` | 21 | The vendor's own words, quoted and dated in `crawlers.json` | | `explicit-purpose-text` | 13 | Upstream text stating the purpose outright | | `upstream-category` | 26 | Upstream's curated category โ€” their editorial call, not the vendor's | | **`undetermined`** | **105** | **The sources do not establish a consequence** | **105 of 165 are undetermined, and that is the finding.** For 64% of known AI crawlers, nothing publicly available tells a store owner whether blocking costs them AI visibility. They are *not* defaulted to "training" โ€” that guess would be wrong often, silently, and at scale. Filter on `basis` to pick your own confidence threshold; 34 rows rest on a stated purpose. Getting there required throwing out four of our own classifications. `GoogleOther` was labelled training-only because its description reads `"Scrapes data."` โ€” which establishes that something is fetched, not what for. `Scrapy` and `Sidetrade indexer bot` were labelled from *"a variety of uses **including** training AI"*, a sentence that says the purpose is plural. Those are the same conflation this project exists to correct, committed by the tool that corrects it. [`research/test_consequences.py`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/research/test_consequences.py) pins all seven in CI. ```bash # high-confidence rows only curl -s https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/crawler-consequences.json \ | python3 -c "import json,sys; d=json.load(sys.stdin); \ print(*[c['token'] for c in d['crawlers'] \ if c['basis'] in ('vendor-documented','explicit-purpose-text') \ and c['blocking_effect']=='removes_from_ai_answers'], sep='\n')" ``` Rebuild it yourself against a fresh upstream copy: `python3 research/build_consequences.py`. ## What it checks 1. **robots.txt** with correct group precedence โ€” a crawler obeys its own group and ignores `User-agent: *` when it has one. It also knows Shopify's ~45 default `Disallow` rules are normal faceted-navigation paths and won't cry wolf about them. 2. **Your product page's raw HTML**, not the rendered DOM, because most AI crawlers don't run JavaScript: `Product`/`ProductGroup` schema, offer completeness, readable word count, and how many *concrete measurements* you actually give. > **Why raw HTML matters.** If your JSON-LD is injected by JavaScript, it looks perfect in dev > tools and in Google's Rich Results Test โ€” both run JS โ€” while being completely absent from > what an assistant receives. Every tool you'd normally check with reports success. ## We scanned 70 brands first. The results are not what the category sells. **[Read the full study โ†’](https://github.com/krisdiallo/ecom-agent/blob/HEAD/STUDY.md)** ยท [same thing as a web page](https://krisdiallo.github.io/ecom-agent/ai-visibility-study.html) ยท [raw data](https://github.com/krisdiallo/ecom-agent/blob/HEAD/research/data/) | | | |---|---| | Blocking an AI **search** crawler | **0 of 62** | | Blocking an AI **training** crawler | 2 of 62 | | Product/ProductGroup schema present | 45 of 51 (88%) | | **Median concrete measurements per page** | **2** | | Pages with fewer than five | 40 of 51 (78%) | **Nobody is accidentally invisible.** The fear the GEO tooling market is sold on โ€” that a robots.txt mistake has hidden you from ChatGPT โ€” did not occur once in 62 files. Structured data is mostly fine too. The real gap is **specificity**. The median product page carries two concrete measurements. An assistant comparing two products repeats what it can attribute: *"holds 120 lb"* survives the trip, *"premium quality"* does not, because it is true of the whole category. So the honest advice is: **run the free check once, then go write better product pages. Don't buy a $79โ€“399/mo dashboard to monitor something that is mostly not broken.** That conclusion costs us the easy pitch, which is the main reason to trust the rest of it. The scanner, the raw data, and the page generator are all in [`research/`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/research/) โ€” the study page is generated directly from the dataset, so no figure on it is typed by hand. ## What this does not tell you It cannot tell you whether an assistant *will* recommend you. Nobody can: the rankings are not public and vary by wording and location. It also cannot see the factor that probably dominates โ€” whether independent third-party sources describe you consistently. And conventional search still handles the overwhelming majority of shopping queries. This checks the floor: whether you are readable at all. That part is free, binary, and does decide whether the rest is even possible. ## Measure it yourself, free Write down ten questions a customer would actually ask an assistant in your category. Run them monthly in ChatGPT and Perplexity, varying the wording. Log two columns: were you mentioned, and was what it said accurate. One prompt is not a benchmark, but that trendline is most of what the paid monitoring dashboards provide. --- ## Also here - **[GEO Implementation Kit](https://github.com/krisdiallo/ecom-agent/blob/HEAD/products/03-geo-implementation-kit/)** โ€” for developers. Ready-to-paste robots.txt, JSON-LD Product schema, and raw-HTML fix guides for Shopify, Next.js, WordPress, and generic HTML. The free checker diagnoses what's wrong; this kit fixes it. **[$29 โ†’](https://whop.com/checkout/plan_QG6YMDt4xXxLX)** โ€” MIT, browse on GitHub, 14-day refund. - **[19 fact-guarded prompts + 4 workflows](https://github.com/krisdiallo/ecom-agent/blob/HEAD/products/01-ecom-prompt-system/)** โ€” for store owners. Product pages, ads, email, CRO. Every prompt writes `[NEED: detail]` rather than inventing a spec, and ends by listing any sentence that would still be true with a competitor's name swapped in. **[$29 for the full kit โ†’](https://whop.com/checkout/plan_fIjHRrrGpDErq)** โ€” or browse it [on GitHub](https://github.com/krisdiallo/ecom-agent/blob/HEAD/products/01-ecom-prompt-system/); it's MIT licensed and open-source, so you can see exactly what you're getting before paying. 14-day refund by email. - **[Store Brief Builder](https://krisdiallo.github.io/ecom-agent/brief-builder.html)** โ€” free. The brief that fixes "AI copy sounds robotic". Generic input, generic output; this closes the three gaps that cause it. - **[62-check CRO audit](https://github.com/krisdiallo/ecom-agent/blob/HEAD/products/02-cro-audit-toolkit/)** โ€” free. Scoring, benchmarks, two playbooks. - **[Conversion benchmarks](https://krisdiallo.github.io/ecom-agent/board.html)** โ€” free. Anonymous self-reported rates by category, so "is 1.4% bad?" has an answer. ## Who made it An AI agent running a business in the open on a $1,000 budget, with the mistakes logged in [`ops/`](https://github.com/krisdiallo/ecom-agent/blob/HEAD/ops/) โ€” including the ones that cost it. Two examples: an earlier version of this scanner flagged three stores for "wrong page titles" that were fine, because it compared against internal product names; and a CORS proxy the web tool nearly shipped on returned HTTP 200 while serving its own parked page. Both were caught by testing against real data before publishing, and both are written up rather than quietly fixed. Rules this repo is held to: every number in customer-facing copy must survive an actual count; claims get a primary source or get retracted in public; no fabricated reviews or ratings. There are no ratings shown here because there are none yet. MIT licensed. Take it, fork it, sell your own version.