AI Crawler Index vs Markdown

markdown_lane_self_report

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"markdown_lane_self_report","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/markdown and the answer is which markdown addresses YOUR client has fetched on this host and which pages it took as HTML instead — each one beside the .md address of the same document — plus this host's own markdown lane measured over 24h and 7d from its published log. THIS IS THE CALL TO MAKE FIRST on this server: it is the only one whose subject is the request you just sent, so there is nothing to look up and no argument to invent. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Every fact is computed from your own headers and from files this host already publishes; nothing is fetched to build it, and no URL of yours is touched. Only this server answers whether your own client has ever taken this host's markdown lane — which .md addresses it fetched, and the markdown address of every page it took as HTML instead — its six siblings answer one other question each about you, same empty arguments, each under the tool named beside its path: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/triage (no_arguments_triage_this_hosts_own_crawler_log), /mcp/robots (no_arguments_lint_this_hosts_robots_txt), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors) on https://www.pathwren.workers.dev; whoami is answered by all seven and is never an error, so one identical body still walks the whole family if you prefer that. Example: the complete call, exactly as written, nothing to fill in — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"markdown_lane_self_report","arguments":{}}} returns your own rows split into markdown and HTML addresses, the .md twin of each page you took as HTML, and how many distinct external clients used this host's markdown lane in the last 24 hours.

whoami

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/markdown and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back describes YOU, the caller — the user-agent you sent, the address you came from, the class our instrument books you as, and whether this host has seen you before. Takes no arguments. Safe to call. Deterministic. Touches no third party. Classifies the request you just sent: the user-agent you claim, the address you came from, the class this host's own instrument books you as, whether we have seen you here before and what you fetched, and whether your client has ever taken this host's markdown lane — which .md addresses it fetched, and the markdown address of every page it took as HTML instead. Every fact comes from the headers on your own request or from a file this host already publishes — nothing is fetched, nothing about you is invented, no argument exists. Example: arguments={} returns your user-agent, your address, the class we book you as and whether we have seen you here before.

example

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/markdown and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back is this server's own worked example — explain_llms_txt, run for real on a canned input from this host's own data — so you see the exact shape of an answer before you send anything of yours. Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — explain_llms_txt on the reference skeleton this server hands out as the answer to every llms.txt finding — parsed here as if you had pasted it, so nothing is fetched and no third party is touched — and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs explain_llms_txt with {"text":"# Your project\n\n> One paragraph: what this is, who it is for, and what it is no and returns its real answer.

check_markdown_lane

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"markdown_lane_self_report","arguments":{}}} — Reads the markdown lane of a URL you name and returns six verdicts with the exact fix for each: /llms.txt, /llms-full.txt, the .md twin of the page, Accept: text/markdown negotiation on the page itself, /sitemap.md, and whether the page ADVERTISES its twin with <link rel="alternate" type="text/markdown"> or the equivalent Link header. Each check reports served / missing / soft-404 / gated / unreachable — a 200 carrying an HTML 404 page is called out as the trap it is, because a client stores it as content. It makes one GET per check to the host you name, identified as pathwren-markdown-lane-check/1.0, and it refuses its own publisher and every ephemeral hostname before opening a socket. Example: arguments={"url":"https://example.com/"} returns the six verdicts for that origin.

explain_llms_txt

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"markdown_lane_self_report","arguments":{}}} — Parses an llms.txt — pasted as text, or fetched from a url you name — into its title, blockquote summary, sections and links, and returns findings by severity with the fix for each: no H1, no summary, links that point at HTML instead of the markdown twin, relative links that break when the file is copied into somebody else's index, a sitemap dump wearing an llms.txt name, HTML tags in the body, duplicate targets, and a missing `## Optional` section on a long list. Returns a ready-to-edit skeleton with every answer. Pasted text is parsed in the request and discarded; nothing is stored. Example: arguments={"url":"https://example.com"} fetches https://example.com/llms.txt and critiques it.