SmartCLI

Read this in: English · 简体中文 · 繁體中文 · 日本語 · 한국어

A local Python toolkit for driving, perceiving, and rendering the terminal — three agent skills over one pluggable PTY + pyte core.

Let an AI drive, perceive, and render real terminal programs. SmartCLI reads the actual screen with a pyte cell model — not a byte pipe — so it knows which menu row is highlighted, presses the right keys, and waits for the screen to settle. Below: it drives the real lazygit TUI end-to-end (arrow-key navigation, opening a commit diff, highlighting a branch) — no script, no mock.

pip install smartcli-toolkit

Requires Python 3.10 or newer. The install includes the shared Python library, the persistent TUI driver, and the stdio MCP server.

Drive something in 30 seconds

Copy-paste this. It starts a real Python REPL under a PTY, waits for the prompt (never a blind sleep), types into it, and reads the screen back:

pip install smartcli-toolkit
SID=$(smartcli-tui start --cmd "python3 -i -q" --cols 80 --rows 24 --json | python3 -c "import json,sys;print(json.load(sys.stdin)['sid'])")
smartcli-tui wait-regex --id $SID ">>> " --timeout-ms 15000
smartcli-tui send-line --id $SID "print(6*7)"
smartcli-tui wait-regex --id $SID "42"          # prints the cell grid it sees
smartcli-tui close --id $SID

On Windows use --cmd "py -i -q". Swap the command for vim, htop or lazygit and the same five verbs drive those too — that is the whole point: wait-regex and friends react to what the screen actually shows, so an agent never guesses whether its keystroke landed.

Want the same thing against a real editor, end to end and verifiable? examples/drive_vim.py drives the actual vim binary — opens a file, appends a line, saves, and then checks the filesystem, not the screen:

python examples/drive_vim.py
#   [OK ] vim painted its screen
#   [OK ] file contents visible on screen
#   [OK ] alternate screen is active
#   [OK ] vim entered insert mode (so G and o both landed)
#   [OK ] typed text appears on screen
#   [OK ] vim restored the main screen on exit
#   [OK ] file on disk really changed

Note the fourth step. It is there because the example itself once sent five keystrokes back to back with nothing between them, and under load vim had not processed G by the time o arrived, so nothing was inserted and the run failed with no useful diagnosis. Confirming insert mode proves both keys landed — the same discipline the tool exists to provide, applied to its own demo.

Run the same file against smartcli-toolkit==0.1.8 and two steps fail — and the file is never saved, because a driver that cannot see the alternate screen mistimes the :wq. That is why the emulation work below matters: a wrong screen model does not error, it silently succeeds at nothing.

Already running an MCP client (Claude Code, Cursor, VS Code)? The same verbs are MCP tools, with the per-session token attached for you:

smartcli-mcp        # stdio MCP server; or `uvx --from smartcli-toolkit smartcli-mcp`

What & why

SmartCLI is a workspace for terminal work that agents and humans both do: driving interactive terminal programs, perceiving what a screen actually shows, and rendering visuals and layouts back out. It is built on one shared, pluggable PTY backend plus a pyte screen model — chosen over screenshot/vision so a single structured screen model feeds both perception (read the screen) and rendering (draw the screen). The PTY layer is intentionally not tmux-bound: local dev runs on Windows via ConPTY (pywinpty), while target programs can run under POSIX ptys or tmux elsewhere. Three skills sit on that core, each a self-contained tool you run in place from the checkout.

Driving a real TUI

The demo above is SmartCLI driving lazygit — a real full-screen curses app — through its perceive → act → confirm loop: it reads the pyte cell grid (which row is selected, the alt-screen diff), moves with arrow keys, opens a commit's diff, and highlights a branch. Captured by driving the actual program in a Linux container, not scripted or mocked. A byte-stream matcher like pexpect can't perceive "which row is highlighted"; a screen model can.

How we know the perception is right. A screen model is only useful if it matches what a real terminal shows, so we measure that instead of asserting it: identical bytes go to a real tmux pane and to our model, and the two cell grids are diffed. Three suites do it — 35 curated cases, a three-way check that only trusts a behaviour when tmux and GNU screen agree, and a generative fuzz over random VT sequences. That campaign found and fixed 12 emulation bugs, including the alternate screen buffer (pyte implements none of modes 1049/1047/47, so a full-screen program's output used to be painted over the main screen and never restored). Scope and remaining edges: LIMITATIONS.md.

Live effects

Real captures of the cmd-art fx engine — each GIF is the actual effect rendered frame-by-frame through the project's own pipeline (no screen recorder). Reproduce any with python -m fx play <name> (see Quickstart).

_{solarsystem — an orrery: planets on elliptical orbits around a pulsing sun}

donut — the classic ASCII torus	fire — demoscene heat field	rain — Matrix digital rain

🌐 Explore the live showcase → — play with the effect engine, drive a menu with arrow keys, and poke the widgets, right in your browser.

Install

Just want the three Claude Code skills? Download one zip, unzip it, done — no git, no pip, no marketplace:

curl -LO https://github.com/dwgx/SmartCLI/releases/latest/download/smartcli-skills.zip

unzip smartcli-skills.zip -d ~/.claude/skills/

That gives you cmd-art, drive-tui and tui-ui (309 KiB total). cmd-art and tui-ui then work with nothing but CPython 3.10+ — verified on a bare virtualenv, all 30 effects and all 17 widgets load. drive-tui additionally needs pyte, which the PyPI install below provides. Or install all three via the plugin marketplace: /plugin marketplace add dwgx/SmartCLI.

Primary — from PyPI (the library, the CLI, and the MCP server):

pip install smartcli-toolkit

Distribution vs import name: the PyPI distribution is smartcli-toolkit (the names smartcli / smart-cli were taken or blocked), but the importable package is smartcli_core. So after pip install smartcli-toolkit you still write from smartcli_core import PtySession.

Alternative — reproduce the full dev environment from a source checkout:

git clone https://github.com/dwgx/SmartCLI SmartCLI
cd SmartCLI
python -m pip install -r requirements.txt

requirements.txt installs pyte, the MCP SDK, and pywinpty on Windows only (POSIX uses the stdlib pty backend). pip install . installs smartcli_core plus the smartcli-tui, smartcli-mcp, and smartcli-toolkit commands. The visual cmd-art and tui-ui skills still run in place from a checkout via python -m fx and python -m ui.

Optional extras (real FIGlet fonts, raster images, authoritative cell widths — all degrade gracefully to stdlib fallbacks when absent):

python -m pip install -r requirements-optional.txt
# or, from the checkout, via pyproject extras:
pip install ".[all]"        # pyfiglet + Pillow + wcwidth
pip install ".[art]"        # pyfiglet only
pip install ".[image]"      # Pillow only  (also: the PNG screenshot harness needs it)
pip install ".[width]"      # wcwidth only

Windows note: set UTF-8 output before running any skill so box-drawing and CJK glyphs encode cleanly (the CLIs also auto-reconfigure stdout, but set this to be safe):

set PYTHONIOENCODING=utf-8