Embodify

What Embodify MCP server does

Embodify MCP server gives an MCP-capable agent a control and observation interface for simulated robots. Supported backends include LIBERO, with 130 tasks across five suites, and RoboDojo, with 54 simulation tasks. The included fake and fake-two-arm backends provide kinematic diagnostics without requiring a simulator, while the remote backend can run another supported backend on a separate machine.

The server keeps the agent's model, memory, skills, and other tools outside the robot-control layer. It does not make LLM calls itself. Episodes, budgets, tool calls, robot states, and camera frames can be recorded for replay and analysis. A browser-based live monitor and frame-by-frame replay are also described in the project documentation.

How it works

An agent communicates with the embodify-mcp process over MCP stdio. The server presents a consistent interface while a backend handles the simulator or diagnostic robot. This means the agent sees the same general tool surface when switching between supported backends.

A typical run starts by listing available tasks, resetting one task, and inspecting the returned camera images and robot state. The agent can then call observe without moving, use move_relative to translate an end effector with an optional rotation, and call set_gripper to open or close a gripper. Two-arm backends can use control_arms to move multiple arms with shared progress. get_session_info reports the current run, task, step budget, and robot state without images. stop_episode ends the run and writes its record.

Each movement call returns actual displacement, remaining error, a stop reason, and fresh camera images. Episode budgets and per-call step limits can be configured and logged. For remote simulation, the project describes SSH or trusted TCP connections, heartbeats for high-latency links, and clean episode abortion when a connection drops.

Setup and configuration

The Embodify MCP server starts with uv and is available from PyPI. Install uv with Python's package installer or its official installer, then register the server with an MCP-capable agent. The repository provides plugin installation commands for Claude Code and Codex; the plugin includes both the MCP server, registered as embodify, and the related skills.

Other agents can register the server using the repository's MCP configuration example. Agents that support SKILL.md folders can also load the skill directories supplied in the repository. The default plugin setup starts on the Fake backend, allowing the tools to be tried without a simulator, GPU, or model. Settings are read from ~/.embodify/config.json.

A standalone smoke check is available through uvx using the embodify-mcp-smoke command. To use a simulator, configure LIBERO locally or connect to an existing simulator according to the backend documentation.

Tools and capabilities

The Embodify MCP server exposes these capabilities:

• Inspect session state, including the run, task, step budget, and robot state.
• Browse the backend task catalogue.
• Start an episode and receive initial camera images.
• Observe camera images and robot state without issuing a movement.
• Move one end effector by a relative translation and optional rotation.
• Open or close a gripper.
• Coordinate multiple arms on two-arm backends.
• Stop an episode and save its record.

The accompanying skills provide guidance for an observe–act loop, maintaining a profile of the robot and cameras, and learning from prior episodes. The repository states that additional perception skills are planned.

Limitations and notes

The current release supports LIBERO and RoboDojo, plus the fake diagnostic backends. RoboTwin and the LeRobot SO-101 arm are identified as upcoming rather than current backends. Other real robots, including xArm, mobile manipulators, humanoids, and dexterous hands, are also listed as future possibilities.

The README names Claude Code and Codex as installation targets and says the server works with MCP hosts generally. It does not document client-specific setup for every MCP application. Real simulator connections may require local simulator setup, a remote machine, or suitable network access. Task success is recorded for evaluators and withheld from the agent during evaluation, and the project supports locked scenes, configurable budgets, and saved episode records for research comparisons.