MCP Turboquant

What ShipItAndPray/mcp-turboquant MCP server does

ShipItAndPray/mcp-turboquant MCP server gives an MCP-compatible agent a workflow for preparing quantized versions of Hugging Face models. It can retrieve basic details such as parameter count, model size, and architecture; inspect which quantization backends are installed; recommend a format and bit width based on available hardware; and perform conversion to GGUF, GPTQ, or AWQ.

The workflow also includes post-conversion perplexity evaluation and an operation for publishing a quantized model to Hugging Face Hub. These capabilities make the server suitable for agents that need to move from model selection to format selection and conversion without requiring a separate quantization command-line utility.

How it works

The server communicates with the MCP client through standard input and output. A client sends a tool request to the Python process, which performs the requested operation in-process and returns the result through MCP. The quantization implementation uses the relevant Python backend packages, including llama-cpp-python, auto-gptq, or autoawq, depending on the selected format and installed extras.

Model information and recommendations do not require the heavy quantization dependencies. Actual conversion and perplexity evaluation do require the appropriate backend support on the machine. The recommendation tool considers the host hardware when suggesting a format and bit setting.

Setup and configuration

Install the package from PyPI with pip install mcp-turboquant, or run it without a separate package installation by using uvx mcp-turboquant. The executable can also be started directly with mcp-turboquant or launched as a Python module with python -m mcp_turboquant.

For conversion, install the extra matching the desired output format:

• mcp-turboquant[gguf] for GGUF conversion
• mcp-turboquant[gptq] for GPTQ conversion
• mcp-turboquant[awq] for AWQ conversion
• mcp-turboquant[all] for all listed quantization backends

In Claude Desktop, add an MCP server entry that runs uvx with mcp-turboquant as its argument. A direct mcp-turboquant command is also supported when the package is installed locally. The same stdio pattern can be used by other MCP clients that support local process servers.

Tools and capabilities

The available tools are:

• info: obtains Hugging Face model parameters, size, and architecture details.
• check: reports which quantization backends are available on the current system.
• recommend: proposes a quantization format and bit width using the machine's hardware.
• quantize: converts a model to GGUF, GPTQ, or AWQ.
• evaluate: runs perplexity evaluation against a quantized model.
• push: uploads a quantized model to Hugging Face Hub.

A typical agent interaction can start with info, use recommend to choose settings, call quantize, evaluate the resulting model, and then use push to publish it.

Limitations and notes

The server does not bundle every quantization backend in its base installation. The info, check, and recommend tools work without those optional dependencies, but conversion and evaluation need the relevant extra installed first. GGUF, GPTQ, and AWQ therefore depend on the local Python environment and the backend selected for the operation. The provided material does not specify authentication or credential configuration for Hugging Face uploads.