🔎 Search & Data Extraction
AI Crawler Index
Read-only MCP server over a public reference index of web crawlers and AI user agents: classify a user-agent, look up a crawler and who runs it, generate robots.txt rules, check an IP, and see what blocking each one costs. Streamable HTTP, no key and no signup; the data is CC0.