geo-inspector-mcp

Inspect any website's AI-search readiness from Claude (or any MCP client): which AI crawlers it blocks, whether it publishes llms.txt, what schema markup it ships, and how its indexing directives are set.

Why this exists

AI assistants are becoming a primary way people find and cite content, and sites signal their intent to AI systems through a handful of plumbing files: robots.txt rules for AI crawlers, the emerging llms.txt standard, schema.org structured data, and meta directives. Checking those by hand means juggling curl, a robots.txt parser in your head, and view-source. This server turns all of it into questions you can just ask Claude.

Quickstart

npx -y geo-inspector-mcp

That is the whole install. Point your MCP client at it:

Claude Code

claude mcp add geo-inspector -- npx -y geo-inspector-mcp

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "geo-inspector": {
      "command": "npx",
      "args": ["-y", "geo-inspector-mcp"]
    }
  }
}

Then ask things like: "Which AI crawlers does nytimes.com block?" or "Does stripe.com publish an llms.txt?"

Tools

Every tool returns a readable summary plus structured JSON (structuredContent) for programmatic use.

Development

npm install
npm test        # vitest unit + integration tests
npm run build   # bundle to dist/
npx @modelcontextprotocol/inspector node dist/index.js   # poke it interactively

Parsers are pure functions with fixture-based tests; all HTTP goes through one capped, redirect-limited fetch helper.

License

MIT