Best Alternatives to AI Crawler Index

Identify AI crawlers by user-agent or IP, verify them against operator ranges, build robots.txt.

Check up to 500 IPs against operator-published crawler ranges; export a firewall allowlist.

Paste access-log lines: every crawler identified, impersonators named, robots.txt and WAF rules.

Lint a robots.txt, test paths against RFC 9309, see which AI crawlers it really blocks.

Markdown

Runtime: Remote (SSE)

Six checks: llms.txt, llms-full.txt, .md twins, Accept: text/markdown, sitemap.md, alt link. Free.

Lint any MCP endpoint you name: handshake, tool schemas, error conformance. Free, no key.

Which discovery documents does a host serve? llms.txt, agent card, owners.json, mcp.json.

PraisonAI

Runtime: npx

AI Agents Framework with Self Reflection and MCP support

Labelhead Artist Momentum

Runtime: npx

Trending hip-hop artist momentum scores across four cultural dimensions.

MediaWiki MCP Server

Runtime: npx

MCP server enabling AI clients to interact with any MediaWiki wiki through standard tools

Affset

Runtime: npx

Run affset from chat: stats, campaigns, zones, payouts, targeting, and team.

Andrea9293 MCP

Runtime: npx

Local-first document management and semantic search for AI coding agents