Web Analyzer MCP

What kimdonghwi94/Web-Analyzer-MCP MCP server does

kimdonghwi94/Web-Analyzer-MCP MCP server provides two MCP tools for working with web pages. url_to_markdown accepts a page URL and produces cleaned Markdown that preserves important text, tables, images, and other structured information. The extraction process is intended to remove common page clutter such as advertisements, navigation elements, duplicate content, and less relevant sections.

web_content_qna accepts a URL and a question. It extracts page content, identifies sections related to the question, and generates an answer with an OpenAI GPT model. This makes the server suitable for page-specific research, documentation lookup, and retrieval-augmented workflows where an agent needs a compact representation of a web page.

How it works

The server is built with FastMCP and runs as a Python module. For page extraction, it validates the URL, loads the page through Selenium, and parses the resulting HTML with BeautifulSoup. A custom scoring process ranks page elements by importance before filtering and converting the selected content to Markdown.

For Q&A, the server divides extracted text into chunks, scores those chunks for relevance to the supplied question, and selects context for answer generation. OpenAI models handle the final response. The README lists GPT-3.5, GPT-4, GPT-4 Turbo, and GPT-5 as model choices, with the configured model selected through an environment variable.

Setup and configuration

Install Python 3.10 or newer and make Chrome or Chromium available for Selenium-based browsing. The package can be installed with pip install web-analyzer-mcp, or the repository can be cloned and installed in editable mode. The documented server command is python -m web_analyzer_mcp.server.

Set OPENAI_API_KEY when using the Q&A tool. OPENAI_MODEL is optional; the documented default is gpt-3.5-turbo. The README shows configurations for Claude Desktop, Cursor, VS Code through the Claude Code extension, and PyCharm with an MCP plugin. The OPENAI_API_KEY value is supplied through the client’s environment configuration rather than as a tool argument.

Tools and capabilities

• url_to_markdown: converts a web page into cleaned, structured Markdown.
• web_content_qna: answers a question using relevant content from a supplied web page.
• Content ranking based on an algorithmic importance score.
• Preservation of selected tables, images, and key text during extraction.
• Model selection through OPENAI_MODEL, including the models listed in the README.

Limitations and notes

kimdonghwi94/Web-Analyzer-MCP MCP server needs Chrome or Chromium for JavaScript-heavy sites. Q&A requires an OpenAI API key; the README does not state that the key is needed for basic Markdown extraction. Automated access may be blocked by some websites, and the server applies rate limiting to prevent abuse.

The project roadmap mentions possible future support for PDFs, videos, additional languages, custom extraction rules, caching, and webhooks. These items should not be treated as current capabilities. The repository states that the project is licensed under MIT.