AI Crawler Index README

The full upstream README, mirrored for reference. Install config, tool schemas, adoption signals and our own overview live on the AI Crawler Index listing page.

We couldn’t automatically pull a README for this listing from pathwren.workers.dev. Open the source repository for setup instructions and documentation.