Web Content & URL Tools

What Web Content & URL Tools does

The Web Content & URL Tools MCP server exposes 26 remote tools for public web and URL analysis. Its web-page tools can turn page content into Markdown, return metadata, extract headings and links, identify technologies, and follow redirects. Additional checks cover TLS certificates, security headers, robots.txt, sitemaps, feeds, domain records, SEO, and broken links.

The service also handles email-domain authentication and identity metadata. It can inspect SPF, DMARC, DKIM, MTA-STS, TLS-RPT, and BIMI, analyze SAML metadata, and check OpenID Connect discovery documents and JWKS data. Separate utilities parse cron expressions, test regular expressions, convert colors, inspect Unicode, convert times across IANA zones, and list time zones. Three tools provide public Hyperliquid perpetuals market data.

How it works

The Web Content & URL Tools MCP server runs remotely as an Apify Standby Actor and communicates through the Streamable HTTP transport. An MCP client sends a tool request to the hosted endpoint, which forwards the request to the Web Content API and returns JSON results.

Two connection paths are documented. Clients can call the Apify Standby URL with an Apify API token in an Authorization header, or use the Apify MCP gateway with OAuth and sign in through a browser. The repository is the source for the hosted Actor rather than a package that clients need to run locally.

Successful calls are billed to the associated Apify account. Page-oriented operations cost $0.003 per call; check and utility operations cost $0.001. Failed requests, including blocked sites, invalid URLs, and timeouts, are not charged.

Setup and configuration

For token-based access, configure the MCP client with the remote URL https://avalonai--web-content-url-tools-mcp.apify.actor/mcp and send the Apify token as a bearer credential. The OAuth route uses https://mcp.apify.com?tools=avalonai/web-content-url-tools-mcp; the README includes a Claude Code command for adding that endpoint with HTTP transport.

The documented clients include Claude Desktop, Claude Code, Cursor, VS Code, and Windsurf. Other clients can use the service if they support remote MCP connections with headers or MCP OAuth, as applicable.

Tools and capabilities

Web Content & URL Tools groups its capabilities into four areas:

• Web analysis: Markdown conversion, metadata, structure, technology detection, redirects, SSL, headers, robots, sitemaps, feeds, domains, SEO, and link checks.
• Email and identity: email authentication scoring, SAML metadata inspection, and OIDC provider checks.
• Developer utilities: cron interpretation, regular-expression testing, color conversion and contrast, Unicode inspection, and time-zone operations.
• Market data: Hyperliquid market snapshots, funding history for up to 30 days, and OHLCV candles from one minute to one week.

Results are designed for programmatic use and may include grades, scores, evidence, warnings, issue lists, timestamps, or structured records depending on the tool.

Limitations and notes

Only public HTTP and HTTPS addresses can be fetched. Private and internal destinations are refused, and some websites block automated access; those requests return a target_blocked result. Pages that render most content in the browser may produce little text and include a warning because the service reads the HTML it receives.

The broken-link checker examines up to 100 links and separates broken, redirected, and bot-blocked results. SPF checking has a ten-lookup limit. Hyperliquid results come from public market data, and the server is not affiliated with Hyperliquid; its market data is not financial advice.