Pyjab MCP

What pyjab-mcp does

The pyjab-mcp MCP server exposes an MCP interface for controlling Java desktop applications built with Swing or AWT. It uses pyjab to read the JVM's accessibility tree through Java Access Bridge, allowing an agent to inspect windows and interact with controls without placing code inside the target application.

The server is intended for Windows and does not alter the application being driven. It does not inject into the JVM, restart the process, or change launch parameters, classpaths, or dependency JARs. JavaFX is outside its supported scope because JavaFX exposes accessibility through UI Automation rather than Java Access Bridge.

How it works

A client starts the server, which attaches to an already-running Java window through the accessibility bridge. java_list_windows identifies available windows, and java_attach_window establishes a driver for the selected one. java_snapshot records the accessible hierarchy; later operations can use handles derived from that snapshot.

Handles describe an element's location as well as its name. Before an action is performed, the server finds the element again and compares its role, name, sibling position, and bounds with the recorded information. If the element has changed, the operation is refused rather than applied to a potentially different control.

Attaching a window does not give it keyboard focus. Use java_focus_window before java_send_keys, and check the reported result because a focus request does not guarantee that the window came forward. A driver binding remains for the life of the process; the server does not provide a session-release tool.

Setup and configuration

Install the package with:

pip install pyjab-mcp

The machine must run Windows with Python 3.10 or newer, a JDK, and pyjab>=1.11.0. Enable Java Access Bridge with the JDK utility, then restart the Java application:

%JAVA_HOME%\bin\jabswitch -enable

The Java application must already be running before the server attaches to it. The repository also publishes .mcpb bundles for supported CPython 3.10–3.13 environments. These bundles do not include Python or a JDK and require 64-bit Windows, with python available on PATH.

Tools and capabilities

The pyjab-mcp MCP server provides these tools:

• Diagnose Java Access Bridge and list or attach to windows.
• Create accessibility snapshots and locate or inspect elements.
• Read element text and retrieve paged JTable data through AccessibleTable.
• Click controls, set text, select values, and fill forms.
• Send keyboard chords to the window with keyboard focus.
• Bring a window forward, wait for conditions, and capture screenshots.

The published tool names are java_diagnostics, java_list_windows, java_attach_window, java_snapshot, java_find, java_get_element, java_get_text, java_get_table, java_click, java_set_text, java_select, java_fill_form, java_send_keys, java_focus_window, java_wait_for, and java_screenshot.

Limitations and notes

This OS Automation server supports Swing and AWT, not JavaFX. It depends on Java Access Bridge being enabled for the application and on the target process exposing the relevant accessibility information. Keyboard chords go to whichever window currently owns keyboard focus, so attachment alone is insufficient.

The server's live tests exercise a separate Swing test application, not every application an agent may encounter. A snapshot handle is positional rather than a permanent object reference, and changes to the target element cause the safety check to stop the action. Table extraction is available for Java JTable controls through the JVM's accessibility interfaces.