Local Llm Worker

What local-llm-worker does

The local-llm-worker MCP server moves selected, high-volume work from a frontier model to a model running on hardware you control. It exposes three main tools:

• offload runs a command or reads a large file, then returns the local model's answer instead of sending the full material to Claude.
• research searches the web, reads result pages, and produces an answer with citations and a source list.
• delegate optionally gives a local model one target file to implement in a temporary Git worktree, using a test as the acceptance check.

This design fits noisy test output, long logs, large documents, and other tasks where the agent needs an answer rather than the complete source material. The server can run several calls concurrently, and the concurrency limit is configurable.

How it works

The server communicates with a configured local model service. Supported backends include Ollama, llama.cpp, LM Studio, vLLM, LocalAI, and endpoints that implement the OpenAI-compatible API. CPU-only operation is supported, although it may be slower. Material is sized for the selected model's context window, with truncation reported when content cannot fit.

For delegate, the local-llm-worker MCP server creates an isolated worktree containing the repository state and uncommitted changes. The model is restricted to one target file, and named test files cannot be selected as targets. After each attempt, the server runs the specified test and supplies parsed assertion failures for supported formats including TAP, pytest, Jest, Vitest, Go, and Cargo. A passing result can copy the file back unless it changed concurrently; failed attempts leave the worktree available and report the final failure.

Setup and configuration

Requirements are Node.js 18 or newer, Git, and a running local model server. The repository can be started directly with Node without installing npm dependencies:

{
  "mcpServers": {
    "local-llm-worker": {
      "command": "node",
      "args": ["/path/to/local-llm-worker/src/index.js"],
      "env": { "LLW_MODEL": "qwen3-coder:30b" }
    }
  }
}

Claude Code users can install the supplied plugin or register the server with claude mcp add. Claude Desktop, Cursor, Windsurf, and other MCP clients can use the stdio configuration above. The plugin can help discover a backend, choose a model, and select which tools are automatically suggested; delegate is opt-in by default.

Configuration can be changed through the configure tool and takes effect on the next call. Settings may also be stored in ~/.config/local-llm-worker/config.json, a project-level .local-llm-worker.json, or environment variables named LLW_<KEY>. Later configuration layers override earlier ones. Important settings include base_url, api, api_key, headers, model, per-tool model overrides, the SearXNG search_url, and the number of research sources.

Tools and capabilities

Use offload for test suites, builds, command output, logs, and files that would be expensive to place in the primary model's context. Use research when a cited web response is preferable to exposing full pages to Claude. Use delegate for narrowly defined implementation tasks where a test and explicit specification can constrain the edit.

The local-llm-worker MCP server keeps API keys out of displayed configuration values, supports extra HTTP headers for gateways, and cleans up stale temporary worktrees. Untracked files larger than 10 MB are excluded from delegate worktrees and reported in the result.

Limitations and notes

Results depend on the configured model, backend, hardware, and task specification. Delegation is deliberately limited to one repository file and should not be used for authentication or money-related code, according to the bundled guidance. The server does not assume a GPU, but CPU execution can take longer. Transport errors are reported separately from model test failures, and a delegate stops after its configured number of attempts.