Macos Vision MCP

What woladi/macos-vision-mcp MCP server does

The woladi/macos-vision-mcp MCP server provides local document OCR, image analysis, and macOS UI inspection through MCP tools. It uses Apple Vision Framework capabilities to process JPG, PNG, HEIC, TIFF, and PDF files on the Mac. OCR results can include reading-order paragraphs, text blocks, line and paragraph identifiers, bounding boxes, and confidence values. PDF analysis supports selecting a starting page and limiting the number of pages processed.

The server also detects faces, reads barcodes and QR codes, identifies document corners, and classifies images into more than 1,000 categories with confidence scores. analyze_document combines document text, layout data, face detections, barcode results, and rectangle information into structured JSON that an agent can use to produce Markdown, HTML, DOCX, or another format.

For Mac UI work, tools can list windows, capture the main display, capture a window or region, read visible text, locate controls by their displayed text, and verify whether text is present or absent. ui_snapshot exposes accessibility-tree information such as element bounds, roles, labels, and states, with optional colour and font data. The server reports click coordinates but does not perform clicks itself.

How it works

The woladi/macos-vision-mcp MCP server runs locally and invokes macOS Vision and screen-access capabilities. Images and documents remain on the Mac during processing. MCP clients receive extracted text, structured JSON, geometry, classifications, or assertion results; the tools do not return image bytes. This makes the server suitable for reducing the amount of document or screen content passed to a model, although the stated token reduction is an example rather than a guaranteed result for every file.

A typical UI workflow is to list windows, locate an element by visible text, pass the returned clickPoint to a separate input automation tool, and then use assert_text to check the result. Matching supports exact, substring, and fuzzy approaches, with near-miss reporting. Screen capture can target an occluded window or an application's frontmost window without requiring the agent to interpret a remote screenshot.

Setup and configuration

Install the published package with:

npx -y macos-vision-mcp

The README identifies macOS 13.0 or later as the supported platform. The server is designed to work offline after installation and does not require an API key. Screen and accessibility features depend on the corresponding macOS permissions; vision_capabilities can report Screen Recording and Accessibility permission state, the macOS version, and connected displays.

Tools and capabilities

• Extract OCR text and structured layout data from images and PDFs.
• Process selected PDF page ranges with start_page and max_pages.
• Detect faces, barcodes, QR codes, document corners, and image categories.
• Capture screens, windows, application windows, and regions while returning file paths and screen coordinates.
• Read screen text locally and find visible UI elements with click coordinates.
• Assert screen text locally and inspect accessibility-tree structure with ui_snapshot.

Limitations and notes

The woladi/macos-vision-mcp MCP server is macOS-specific and depends on Apple Vision Framework functionality. It reports where an element can be clicked but does not control the pointer or keyboard, so a separate input driver is needed for actions. Screen and accessibility operations may not work until macOS grants the required permissions.

The server returns structured representations rather than screenshot image bytes. Agents that need visual details not captured by OCR or accessibility data may therefore need another approach. Processing is local, but the resulting extracted text and JSON can still be sent onward by the MCP client to its model provider.