Web Hygiene MCP

What Web Hygiene MCP MCP server does

Web Hygiene MCP MCP server lets an AI agent inspect publicly available website information at request time. It works with robots.txt, llms.txt, XML or other sitemaps, RSS, Atom, and JSON Feed endpoints, as well as public pages and URLs supplied to the tools.

The server provides seven tools. site_overview reports selected robots.txt directives, declared sitemaps, AI crawler instructions, llms.txt availability, sampled sitemap counts, and detected feeds. list_site_urls returns sitemap URLs with lastmod values and supports inclusion or exclusion patterns. check_url reports the HTTP status, redirect path, final URL, response time, and content type.

check_links checks groups of URLs or sitemap entries for broken responses, server errors, timeouts, SSL problems, redirects, and slow pages. discover_feeds locates website feeds, while read_feed normalizes feed entries into fields such as title, link, UTC date, author, summary, and enclosures. verify_citations evaluates cited URLs and can compare a supplied quotation with the page content.

How it works

Web Hygiene MCP MCP server runs remotely as a Streamable HTTP service on Apify rather than as a local process. Connect an MCP client to https://bruco3--web-hygiene-mcp.apify.actor/mcp and send an Authorization: Bearer header containing an Apify API token.

Requests fetch the target site or page during the tool call. Citation verification can classify sources as verified, mismatched, missing, redirected to a home page, paywalled, or blocked, among other outcomes. When enabled, it can also look for an Internet Archive copy of a dead link. The service requests public files and pages with a clear user agent; the repository states that it does not collect or store personal data.

Setup and configuration

Create an Apify account and obtain an API token.

For an MCP client that supports Streamable HTTP, configure the server URL and bearer token. A compatible configuration has the following shape:

{
  "mcpServers": {
    "web-hygiene": {
      "url": "https://bruco3--web-hygiene-mcp.apify.actor/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_APIFY_API_TOKEN"
      }
    }
  }
}

Claude Code users can install the documented plugin with /plugin marketplace add Luca-003/web-hygiene-claude-plugin, followed by /plugin install web-hygiene@luca-003. That plugin supplies verify-sources and site-preflight skills.

Limitations and notes

Web Hygiene MCP MCP server operates on public resources named by the caller and on public discovery files published by sites. It is not a whole-site crawler or a substitute for private-content access. Tool limits apply: URL listing returns up to 2,000 entries, bulk link checks up to 300, feed reads up to 100 items, and citation verification up to 50 citations.

Results describe what live requests observe at call time, so response status, redirects, page speed, feed contents, and accessible text can change. A site may still block requests, require payment, fail TLS checks, time out, or redirect.

The repository contains registry metadata and documentation; the running service is hosted on Apify. Related Apify actors are listed for sitemap extraction, broken-link checking, feed monitoring, and citation or link verification when a separate whole-site or whole-document job is needed.