Skip to main content
AllMCPs
BrowseBestCategoriesStackCompareToolsGuidesBlog
Log in Submit MCP

Stay in the loop

Get new MCP servers and top picks in your inbox.

AllMCPs

The open directory for discovering and installing Model Context Protocol servers.

AllMCPs on GitHub (opens in a new tab)
Launched onTiny Startupstinystartups.com
Explore
  • Browse servers
  • Best MCP servers
  • Categories
  • MCP clients
  • Agent prompts
  • Stack Builder
  • Compare servers
  • Random discovery New
  • Submit a server
  • Pricing & Boost Boost
Learn
  • Guides hub
  • What is MCP?
  • Install guide
  • Build an MCP server
  • Deploy an MCP server
  • Security guide
  • Troubleshooting
  • MCP for SEO & AEO
  • Protocol versioning
  • Blog & updates
Tools
  • All developer tools
  • Config generator
  • Config validator
  • Config auditor
  • MCP playground
  • Token calculator
  • OpenAPI β†’ MCP
  • Badge generator
For agents
  • REST API docs
  • Trust & traffic Live
  • Remote MCP server SSE β†— (opens in a new tab)
  • llms.txt β†— (opens in a new tab)
  • Catalog JSON β†— (opens in a new tab)
Company
  • About
  • Advertise Sponsor
  • Contact
  • GitHub β†— (opens in a new tab)
  • Terms
  • Privacy
AllMCPs VerifiedAllMCPs VerifiedFeatured on Nick LaunchesFeatured on Nick LaunchesLaunch Llama NewsletterLaunch Llama NewsletterVerified DR - allmcps.comVerified DR - allmcps.comFeatured on SaaSGrowFeatured on SaaSGrowFeatured on Twelve ToolsFeatured on Twelve ToolsFeatured on Saaspa.geFeatured on Saaspa.geFeatured on Findly.toolsFeatured on Findly.toolsFeatured on Startup FameFeatured on Startup FameFeatured on LaunchKiwiFeatured on LaunchKiwiFeatured on ScrollLaunchFeatured on ScrollLaunchFeatured on DailyPingsFeatured on DailyPingsFazier badgeFazier badgeFeatured on NewTool.siteFeatured on NewTool.siteFeatured on saasfame.comFeatured on saasfame.comDR Checker - Domain RatingDR Checker - Domain RatingListed on Turbo0Listed on Turbo0Launched on LaunchBoard - Product Launch PlatformLaunched on LaunchBoard - Product Launch PlatformList on SimilarlabsList on Similarlabshttps://codetrendy.comhttps://codetrendy.comListed on DevTool.ioFeatured on BuildlistFeatured on BuildlistLaunched on Tiny StartupsFeatured on ShowMeBestAIFeatured on ShowMeBestAIFind us on LaunchZoneFind us on LaunchZoneAllMCPs VerifiedAllMCPs VerifiedFeatured on Nick LaunchesFeatured on Nick LaunchesLaunch Llama NewsletterLaunch Llama NewsletterVerified DR - allmcps.comVerified DR - allmcps.comFeatured on SaaSGrowFeatured on SaaSGrowFeatured on Twelve ToolsFeatured on Twelve ToolsFeatured on Saaspa.geFeatured on Saaspa.geFeatured on Findly.toolsFeatured on Findly.toolsFeatured on Startup FameFeatured on Startup FameFeatured on LaunchKiwiFeatured on LaunchKiwiFeatured on ScrollLaunchFeatured on ScrollLaunchFeatured on DailyPingsFeatured on DailyPingsFazier badgeFazier badgeFeatured on NewTool.siteFeatured on NewTool.siteFeatured on saasfame.comFeatured on saasfame.comDR Checker - Domain RatingDR Checker - Domain RatingListed on Turbo0Listed on Turbo0Launched on LaunchBoard - Product Launch PlatformLaunched on LaunchBoard - Product Launch PlatformList on SimilarlabsList on Similarlabshttps://codetrendy.comhttps://codetrendy.comListed on DevTool.ioFeatured on BuildlistFeatured on BuildlistLaunched on Tiny StartupsFeatured on ShowMeBestAIFeatured on ShowMeBestAIFind us on LaunchZoneFind us on LaunchZone
Β© 2026 Jackalope Digital LLC. All rights reserved.
  1. Home
  2. πŸ’» Developer Tools
  3. Crawler Log Triage
C
Health: ActiveRecent health check succeeded.Last checked 9/7/2026, 4:17:01 PM

Crawler Log Triage

User RatingsBe the first to rate and review this MCP server! Enrichment pendingWe haven’t run our AI enrichment pass on this listing yet, so the overview, use cases, and FAQ below may be sparse or missing. We work through the catalog over time β€” check back soon.
View Repository

Paste access-log lines: every crawler identified, impersonators named, robots.txt and WAF rules.

Quick Install

Automated & IDE Setup

Copy the AI prompt to install this server into Claude Code, Cursor, or another agent β€” or use 1-click editor setup below.

Add to CursorAdd to VS Code
Manual Client & Custom JSON ConfigExpand JSON β–Ύ

Client Config & Setup

Remote HTTP
Choose your client or environment
Target File:~/Library/Application Support/Claude/claude_desktop_config.json
claude_desktop_config.json
{
  "mcpServers": {
    "crawler-log-triage": {
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp-triage.html"
    }
  }
}

πŸ’‘ Paste the JSON block into your client's configuration file under mcpServers, then restart the application.

Install Tool Schemas (9) Directory Badge Claim listing AlternativesπŸ’» More in Developer Tools

Capabilities & Tool Schemas (9) ~2.9k tokensApproximate context cost of this server’s tool schemas (~4 chars/token), before any tool is called. Actual usage depends on your client and model.Verified live Verified liveCaptured by calling this server’s live tools/list endpoint.

Inspect callable tools, capabilities, and parameters exposed to AI agents by Crawler Log Triage.

no_arguments_triage_this_hosts_own_crawler_log

TAKES NO ARGUMENTS. POST {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_triage_this_hosts_own_crawler_log","arguments":{}}} to https://www.pathwren.workers.dev/mcp/triage β€” the answer is the triage of THIS host's own published request log β€” every operator in it run through the same parser, the same crawler index and the same operator-prefix verification that triage_log applies to a file you paste, rolled up by operator, by category and by crawler, with the share no index entry matches at all and the browser-shaped strings named separately. There is nothing to fill in: the input schema is literally empty, `arguments: {}` and no `arguments` key at all both work, and the subject is a file this host already publishes, so the answer does not depend on you at all. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Nothing is fetched to build it β€” no request leaves this edge, and none is made to you. The other zero-argument call on this server is triage_my_request, same empty arguments, which answers your own request triaged as one line of an access log β€” the crawler this host's index identifies from your user-agent, its operator and category, and whether the address you came from verifies against that operator's published prefixes. whoami and example are here too and take nothing either. Every other tool on this server wants a file pasted in; this one wants nothing. The siblings answer one question each under the tool named beside them: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/robots (no_arguments_lint_this_hosts_robots_txt), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors), /mcp/markdown (markdown_lane_self_report). Example: the complete call, exactly as written, nothing to fill in β€” {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_triage_this_hosts_own_crawler_log","arguments":{}}} returns the rollup by operator, category and crawler over this host's own published window, the cost of blocking each crawler identified, the requests that match no entry in the index, and the robots.txt those findings would generate.

triage_my_request

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"triage_my_request","arguments":{}}} β€” POST that exact body to https://www.pathwren.workers.dev/mcp/triage and the answer is your own request triaged as one line of an access log β€” the crawler this host's index identifies from your user-agent, its operator and category, and whether the address you came from verifies against that operator's published prefixes. THIS IS THE CALL TO MAKE FIRST on this server: it is the only one whose subject is the request you just sent, so there is nothing to look up and no argument to invent. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Every fact is computed from your own headers and from files this host already publishes; nothing is fetched to build it, and no URL of yours is touched. Only this server answers your own request triaged as one line of an access log, by the same parser and the same index that read a whole file β€” its six siblings answer one other question each about you, same empty arguments, each under the tool named beside its path: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/robots (no_arguments_lint_this_hosts_robots_txt), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors), /mcp/markdown (markdown_lane_self_report) on https://www.pathwren.workers.dev; whoami is answered by all seven and is never an error, so one identical body still walks the whole family if you prefer that. Example: the complete call, exactly as written, nothing to fill in β€” {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"triage_my_request","arguments":{}}} returns the single log line built from your address and your user-agent, the triage row for it, and the same rollup triage_log returns for a whole file.

whoami

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} β€” POST that to https://www.pathwren.workers.dev/mcp/triage and the answer is the user-agent you sent, the class this host's own instrument books it as, whether we have logged you here before β€” and the part only this server can answer: the request you just made, run through triage_log as a single log line, naming the crawler it identifies, its operator and category, and whether your address verifies against that operator's published prefixes. Takes no arguments. Safe to call. Deterministic. Touches no third party. Nothing is fetched to build it: every fact comes from the headers on your own request or from a file this host already publishes. Only this server answers the last part β€” its five siblings answer one other question each about you, same empty arguments, each under the tool named beside its path: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/robots (no_arguments_lint_this_hosts_robots_txt), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors), /mcp/markdown (markdown_lane_self_report) on https://www.pathwren.workers.dev. Example: arguments={} returns your user-agent, your address, the class we book you as, whether we have seen you here before, and your own request triaged as one line of an access log, by the same parser and the same index that read a whole file.

example

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} β€” POST that exact body to https://www.pathwren.workers.dev/mcp/triage and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back is this server's own worked example β€” triage_log, run for real on a canned input from this host's own data β€” so you see the exact shape of an answer before you send anything of yours. Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end β€” triage_log on three lines in three of the five formats this parser accepts: an address Google publishes, an address in no published range claiming to be the same crawler, and a `uniq -c` count β€” all of it constants in this file, checked against our own mirror, with nothing fetched from anyone β€” and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs triage_log with {"log":"66.249.66.1 Googlebot/2.1\n203.0.113.9 Googlebot/2.1\n412 GPTBot/1.2"} and returns its real answer.

triage_log

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_triage_this_hosts_own_crawler_log","arguments":{}}} β€” Paste log lines β€” combined, JSON, `uniq -c`, `<ip> <ua>` or bare user-agents, auto-detected β€” for per-line crawler, operator, purpose and cost of blocking, a rollup by operator and category, and every line whose address contradicts its claim. Log text, never a URL. Example: log='66.249.66.1 Googlebot/2.1' returns Googlebot, Google, search, verified.

find_impersonators

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_triage_this_hosts_own_crawler_log","arguments":{}}} β€” Only the lines claiming a crawler whose operator publishes address ranges, from an address in none of them β€” 1987 IPv4 and 1062 IPv6 prefixes, 15 sources. Reverse-DNS operators come back with the command to run: this server makes no outbound request. Example: log='203.0.113.9 Googlebot/2.1' returns one impersonation.

Documentation Overview

Crawler Log Triage is a πŸ’» Developer Tools MCP server listed on AllMCPs. Paste access-log lines: every crawler identified, impersonators named, robots.txt and WAF rules.

Read the full README β†’View source on pathwren.workers.dev β†’

Related MCP Servers

View all in Developer Tools View all alternatives
  • C
    Crawler IP Verifier

    Check up to 500 IPs against operator-published crawler ranges; export a firewall allowlist.

    πŸ’» Developer Tools0 views
    Compare vs Crawler IP Verifier β†’
  • R
    Robots Policy Lint

    Lint a robots.txt, test paths against RFC 9309, see which AI crawlers it really blocks.

    πŸ’» Developer Tools0 views
    Compare vs Robots Policy Lint β†’
  • A
    Agent Discovery Doctor

    Which discovery documents does a host serve? llms.txt, agent card, owners.json, mcp.json.

    πŸ’» Developer Tools0 views
    Compare vs Agent Discovery Doctor β†’
  • Markdown logoMarkdown

    Six checks: llms.txt, llms-full.txt, .md twins, Accept: text/markdown, sitemap.md, alt link. Free.

    πŸ’» Developer Tools0 views
    Compare vs Markdown β†’

Reviews

No reviews yet β€” be the first to share how this listing worked for you.

Frequently Asked Questions about Crawler Log Triage

Add the following block to your claude_desktop_config.json under mcpServers: "mcpServers": { "crawler-log-triage": { "command": "npx", "args": ["-y", "Crawler Log Triage"] } }

AllMCPs Directory Badge

Full Badge Customizer

Showcase your server listing on GitHub or your project documentation. Embed this dynamic SVG badge to highlight official listing status and live engagement.

Badge Style:
Live Dynamic SVG PreviewCrawler Log Triage AllMCPs Directory Badge
Markdown (GitHub README)
[![AllMCPs](https://allmcps.com/api/badge/crawler-log-triage?style=directory)](https://allmcps.com/mcp/crawler-log-triage)
HTML Embed
<a href="https://allmcps.com/mcp/crawler-log-triage"><img src="https://allmcps.com/api/badge/crawler-log-triage?style=directory" alt="Crawler Log Triage on AllMCPs" /></a>

Technical Specs & Signals

CategoryπŸ’»Developer Tools
More technical detailsExpand β–Ύ
TransportSSE (Remote)
RuntimeNode.js
Last updatedSep 7, 2026
Views0
Unique ViewsTotal visits recorded for this listing page on AllMCPs.
Installs0
Installs & Copy ActionsTotal times users copied install commands or configuration snippets for this server.
55Quality signal: Good Β· 55/100How this signal is calculated β–Ύ
Server availability25/25
Verified ownership0/20
Documentation & tools29/30
Adoption & activity1/15
Community engagement0/10

A guidance signal from public completeness & health data β€” not a user rating. New listings start lower and rise as they add docs, get verified, and grow adoption. Signals we can't observe for a listing are skipped, not counted against it.

β˜… FeaturedAllMCPs Server logo

AllMCPs Server

The official MCP server for AllMCPs.com - submit and manage tools directly from your AI. The open directory for MCP servers. Connect Claude, Cursor, Windsurf, and AI agents to databases, tools, files, and APIs. Explore 10,000+ servers. AllMCPs is the premier, open directory for discovering, evaluating, and installing Model Context Protocol (MCP) servers to equip AI agents and LLMs with real-world superpowers.

Explore Server β†’

Own this project?

This directory is pre-filled from public sources. Claim via GitHub README, site badge, or DNS TXT to unlock edit access and the Official badge and attach your website β€” proof is checked automatically, then reviewed by our team.

Free dofollow backlink: add your website and place the AllMCPs badge on it β€” no claim needed. We detect it automatically and keep it verified as long as the badge stays live.

Claim & get free dofollow

Share & Embed

Add our SVG badge (dark/light directory styles) or embeddable widget to your site.

Explore more

More in πŸ’» Developer Tools β†’Best MCP servers for Developers β†’Alternatives to Crawler Log Triage β†’Install in Claude DesktopInstall in CursorInstall in VS Code