Swing MCP

What Swing MCP MCP server does

Swing MCP MCP server lets an MCP client inspect and control Java desktop user interfaces built with Swing or JavaFX. It can work with applications using either toolkit, including applications that combine both. The available workflows cover discovering the UI, clicking controls, entering text, completing forms, and interacting with tables, trees, menus, and dialogs.

The server is intended for software that is otherwise difficult for an AI assistant to reach, such as internal business tools, ERP clients, point-of-sale systems, laboratory software, and logistics applications. It operates through the existing interface rather than requiring changes to the target application.

How it works

The MCP-facing component uses stdio transport. When a client calls launch_app, the server starts a JVM with the agent already loaded. attach_to_app instead loads the agent into an existing JVM using its process ID. The agent opens a loopback-only JSON line protocol socket and reports the selected connection details through a response file.

Commands from the server are sent to the agent and executed on the owning toolkit's UI thread: Swing operations run on the Event Dispatch Thread, while JavaFX operations run on the JavaFX Application Thread. Component identifiers include a toolkit prefix, such as comp- for Swing and fx- for JavaFX, so controls remain distinguishable in mixed-toolkit applications.

Tool metadata includes MCP annotations for titles, read-only behavior, destructive behavior, idempotence, and open-world operations. Clients that use these annotations can treat read-only actions differently from operations such as stop_app, close_window, and evaluate_java.

Setup and configuration

Swing MCP MCP server requires JDK 21 or newer. The project can be built with Maven using mvn verify; GUI tests need a display environment, while CI runs them with Xvfb. Users who do not want to build can install the released swing-mcp.mcpb bundle through a supported client such as Claude Desktop.

For a manual stdio registration, run the server JAR with Java and set SWING_MCP_AGENT_JAR to the path of the agent JAR. After registration, launch either a target application or the included Swing and JavaFX demo applications. A practical first sequence is to launch or attach, call take_snapshot to obtain component UIDs, then use interaction tools with those identifiers.

The agent port range is controlled by the swing.mcp.agent-port-min..max configuration properties. The agent binds only to the loopback interface. Arbitrary Java execution is off by default and requires swing.mcp.evaluate.enabled=true to enable evaluate_java.

Tools and capabilities

The documented examples identify these capabilities:

• Start an application or connect to an existing JVM with launch_app and attach_to_app.
• Inspect the current interface with take_snapshot.
• Click controls and enter values with click and fill.
• Choose values and work with common structured UI elements such as tables and trees.
• Operate menus and dialogs, including controlled application and window actions.
• Run Java evaluation only when the explicit configuration setting allows it.

Swing MCP MCP server exposes one tool set across Swing and JavaFX; the client does not need to select the toolkit before issuing commands.