Mtehabsim/ScreenPilot
š š - enables AI to fully control and access GUI interactions by providing tools for mouse and keyboard, ideal for general automation, education, and experimentation.
Quick Install
{
"mcpServers": {
"mtehabsim-screenpilot": {
"command": "npx",
"args": [
"-y",
"mtehabsim-screenpilot"
]
}
}
}Using an AI coding agent (Claude Code, Cursor, etc.)? Copy a ready-made prompt that tells it to fetch the setup instructions and install this server for you.
Documentation Overview
ScreenPilot
MCP server to let LLM take full control on your device by providing screen automation toolkit for controlling and interacting with graphical user interfaces. Good for automation, education and having fun.
Main Features
- š· Screen capture and analysis
- š±ļø Mouse control (clicking, positioning)
- āØļø Keyboard input (typing, key presses, hotkeys)
watch demo
https://github.com/user-attachments/assets/c18380c0-b3dd-4b7c-925d-28ef205ca11f
Installation
- Install python 3.12
- Clone the repository:
git clone https://github.com/Mtehabsim/ScreenPilot.git - create virtiual environment
python -m venv venv
- activate the env
venv\Scripts\activate
- Install the required packages:
pip install -r requirements.txt - Open Claude AI desktop
- file -> settings -> developer -> edit config
- open config file and paste this
{
"mcpServers": {
"device-controll": {
"command": "pathToEnv\\venv\\Scripts\\python.exe",
"args": [
"pathToProject\\ScreenPilot\\main.py"
]
}
}
}
-
Replace Ā Ā Ā Ā "pathToEnv\venv\Scripts\python.exe" ā with the full path to your python.exe Ā Ā Ā Ā "pathToProject\ScreenPilot\main.py" ā with the full path to your main.py file
-
Save the config file.
-
Open Claude AI Desktop.
-
Go to File ā Exit
-
You can now open Claude AI Desktop and enjoy ScreenPilot.
Available Tools
- Screen Capture: Take screenshots and get screen information
- Mouse Control: Move the mouse and perform clicks
- Keyboard Actions: Type text, press keys, and use hotkey combinations
- Scrolling: Scroll in different directions and to specific positions
- Element Detection: Check if elements exist on screen and wait for them to appear
- Action Sequences: Perform multiple actions in sequence
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.