Isaac Sim MCP

What the omni-mcp/isaac-sim-mcp MCP server does

The omni-mcp/isaac-sim-mcp MCP server adds natural-language control to NVIDIA Isaac Sim. Its repository contains both an Isaac Sim extension and a Python MCP server. Together, they let an MCP client send scene and simulation operations to a running Isaac Sim instance.

Supported workflows include creating physics scenes, placing supported robots, manipulating primitives through Omni Kit commands, and running Python code inside Isaac Sim. The examples also cover arranging multiple robots, changing lighting, adding obstacles, searching for USD assets, and generating Beaver3D objects from image URLs. The README identifies NVIDIA Isaac Sim 4.2.0 or newer as a requirement.

How it works

The extension is loaded by Isaac Sim with the isaac.sim.mcp_extension extension ID. After startup, it exposes a local service on localhost:8766 by default. Separately, the Python MCP server is launched with uv run and configured in Cursor as an MCP server.

An agent can first call get_scene_info to check that the extension is reachable and obtain basic scene information. The documented workflow then uses create_physics_scene before adding robots. For more involved operations, the agent can submit Python through execute_script; the README recommends reviewing generated code in chat before allowing execution. Cursor Agent mode is used in the supplied examples.

Setup and configuration

Use Python 3.9 or newer, NVIDIA Isaac Sim 4.2.0 or newer, and the uv/uvx tools. The README also instructs users to install mcp[cli] into the base environment. Clone the repository, point Isaac Sim's extension folder at the checkout, and start Isaac Sim with the extension enabled.

The MCP process can be tested with:

uv run /home/ubuntu/Documents/isaac-sim-mcp/isaac_mcp/server.py

The path is an example and must match the local checkout. Cursor configuration uses an MCP entry whose command runs that same server file through uv run. The MCP inspector can be started with uv run mcp dev and opens a debug page at http://localhost:5173.

Beaver3D and NVIDIA-related functionality requires the environment variables BEAVER3D_MODEL, ARK_API_KEY, and NVIDIA_API_KEY. The README describes these as setup values for Beaver3D and NVIDIA services.

Tools and capabilities

The documented MCP tools are:

• get_scene_info: checks the extension connection and returns basic scene details.
• create_physics_scene: creates a physics scene with objects, a floor option, gravity, and a scene name.
• create_robot: places a franka, jetbot, carter, g1, or go1 robot at supplied coordinates.
• omni_kit_command: sends an Omni Kit command with a command name and primitive type.
• execute_script: runs arbitrary Python code in Isaac Sim.

The extension also supports natural-language examples for robot movement, lighting changes, obstacle navigation, custom scene creation, and code-assisted simulation construction.

Limitations and notes

The README specifically documents Cursor as the MCP integration client; compatibility with other clients is not established there. Isaac Sim must be installed and running with the extension enabled before scene operations can work. execute_script accepts arbitrary Python, so generated code should be inspected before execution. The standard robot creation tool is limited to the robot types listed above, while additional behavior depends on custom scripts or Omni Kit commands. External Beaver3D and NVIDIA features depend on their corresponding credentials and configuration.