Xcomet MCP Server

xCOMET MCP Server

日本語版 README はこちら

⚠️ This is an unofficial community project, not affiliated with Unbabel.

Translation quality evaluation MCP Server powered by xCOMET (eXplainable COMET).

🎯 Overview

xCOMET MCP Server provides AI agents with the ability to evaluate machine translation quality. It integrates with the xCOMET model from Unbabel to provide:

• Quality Scoring: Scores between 0-1 indicating translation quality
• Error Detection: Identifies error spans with severity levels (minor/major/critical)
• Batch Processing: Evaluate multiple translation pairs efficiently (optimized single model load)
• GPU Support: Optional GPU acceleration for faster inference

graph LR
    A[AI Agent] --> B[Node.js MCP Server]
    B -- stdio JSON-RPC --> C[Python Worker]
    C --> D[xCOMET Model<br/>Persistent in Memory]
    D --> C
    C --> B
    B --> A

    style D fill:#9f9

🔧 Prerequisites

Python Environment

• Python 3.9 - 3.12 recommended (3.13+ is not yet supported by xCOMET dependencies)

xCOMET requires Python with several packages. We recommend using a virtual environment:

# If using uv (recommended - auto-downloads the correct Python version)
uv venv ~/.xcomet-venv --python 3.12
source ~/.xcomet-venv/bin/activate
uv pip install "unbabel-comet>=2.2.7,<3.0"

# Or using standard venv (requires Python 3.9-3.12 already installed)
python3 -m venv ~/.xcomet-venv
source ~/.xcomet-venv/bin/activate  # Windows: ~/.xcomet-venv\Scripts\activate
pip install "unbabel-comet>=2.2.7,<3.0"

Why Python 3.9-3.12? unbabel-comet declares numpy = "^1.20.0", so it resolves numpy 1.x. The last numpy 1.x release, 1.26.4, ships wheels for cp39-cp312 only. On Python 3.13 or later, pip has to build numpy from source.

Note (v0.5.0+): The Python worker now talks to Node.js over stdin/stdout (line-delimited JSON-RPC). FastAPI, uvicorn, and pydantic are no longer required — only unbabel-comet is.

Note: When using with Claude Desktop or other MCP hosts, set XCOMET_PYTHON_PATH to point to the venv Python (see Configuration).

Model Download

Important: XCOMET-XL and XCOMET-XXL are gated models on HuggingFace. You must:

1. Create a HuggingFace account

2. Visit Unbabel/XCOMET-XL and request access

3. Authenticate, either via the CLI:

   bashCopy

   source ~/.xcomet-venv/bin/activate
   hf auth login
   

   (huggingface-cli login still works but prints a deprecation warning since huggingface_hub 0.34; hf is the current command.)

   Or by setting HF_TOKEN in the MCP host's env block, which is the option when the host launches the server in an environment where no CLI login has been performed:

   jsonCopy

   "env": {
     "XCOMET_PYTHON_PATH": "~/.xcomet-venv/bin/python3",
     "HF_TOKEN": "hf_..."
   }
   

   huggingface_hub reads HF_TOKEN first and falls back to the token file written by hf auth login.

Unbabel/wmt22-comet-da does not require authentication (but requires reference translations).

After authentication, download the model (~14GB for XL, ~42GB for XXL):

source ~/.xcomet-venv/bin/activate
python -c "from comet import download_model; download_model('Unbabel/XCOMET-XL')"

Where the model is stored

Not in the virtualenv. The venv holds the Python packages; the model weights go to the huggingface_hub cache, which is a separate directory shared by every tool on the machine that pulls from the Hub.

~/.xcomet-venv/                              ← Python packages only
└── lib/python3.x/site-packages/
    ├── comet/                               unbabel-comet itself
    └── torch/  transformers/  ...           its dependencies

~/.cache/huggingface/                        ← the model weights live here
└── hub/
    └── models--Unbabel--XCOMET-XL/
        ├── blobs/                           the actual ~14GB checkpoint
        └── snapshots/<revision>/
            ├── checkpoints/model.ckpt       what download_model() returns
            └── hparams.yaml

download_model() passes cache_dir=None to snapshot_download(), so huggingface_hub picks the location: HF_HUB_CACHE, which defaults to HF_HOME/hub, where HF_HOME defaults to $XDG_CACHE_HOME/huggingface (~/.cache/huggingface when XDG_CACHE_HOME is unset).

Three consequences worth knowing:

• Rebuilding or deleting the venv does not re-download the model.
• Several venvs, and other Hub-based tools, share the same copy.
• The size of the venv directory does not account for the 14GB. Use hf cache scan to see what is actually on disk, and hf cache delete to remove a revision.

To put the checkpoint somewhere else — a larger volume, a shared drive — set XCOMET_SAVING_DIRECTORY (v0.7.0+) or the standard HF_HOME. Both are read at download time, so a model already downloaded to the default location is not moved; it is downloaded again into the new one.

Node.js

• Node.js >= 22.0.0 (matches engines.node in package.json; CI runs on 22 and 24)
• npm or yarn

📦 Installation

Note: If you just want to use xCOMET MCP Server, you do not need to clone this repository. Install the Python environment and model (see Prerequisites), then use npx (see Usage). The section below is for contributors and local development only.

Local Development

For contributors and local development:

# Clone the repository
git clone https://github.com/shuji-bonji/xcomet-mcp-server.git
cd xcomet-mcp-server

# Set up Python virtual environment and install dependencies
uv venv .venv --python 3.12    # or: python3 -m venv .venv
source .venv/bin/activate
pip install -r python/requirements.txt

# Install Node.js dependencies and build
npm install
npm run build

🚀 Usage

With Claude Desktop (npx)

Add to your Claude Desktop configuration (claude_desktop_config.json):

{
  "mcpServers": {
    "xcomet": {
      "command": "npx",
      "args": ["-y", "xcomet-mcp-server@latest"],
      "env": {
        "XCOMET_PYTHON_PATH": "~/.xcomet-venv/bin/python3"
      }
    }
  }
}

Tip: If you installed Python packages system-wide or use pyenv, XCOMET_PYTHON_PATH may be omitted (auto-detection will find it). See Python Path Auto-Detection for details.

With Claude Code

claude mcp add xcomet --env XCOMET_PYTHON_PATH=~/.xcomet-venv/bin/python3 -- npx -y xcomet-mcp-server@latest

Global Installation

If you prefer installing globally:

npm install -g xcomet-mcp-server

Then configure:

{
  "mcpServers": {
    "xcomet": {
      "command": "xcomet-mcp-server",
      "env": {
        "XCOMET_PYTHON_PATH": "~/.xcomet-venv/bin/python3"
      }
    }
  }
}

Local Development Build

If you cloned and built the repository locally (see Installation):

{
  "mcpServers": {
    "xcomet": {
      "command": "node",
      "args": ["/path/to/xcomet-mcp-server/dist/index.js"],
      "env": {
        "XCOMET_PYTHON_PATH": "~/.xcomet-venv/bin/python3"
      }
    }
  }
}

🛠️ Available Tools

xcomet_evaluate

Evaluate translation quality for a single source-translation pair.

Parameters:

Example:

{
  "source": "The quick brown fox jumps over the lazy dog.",
  "translation": "素早い茶色のキツネが怠惰な犬を飛び越える。",
  "source_lang": "en",
  "target_lang": "ja",
  "use_gpu": true
}

Response:

{
  "score": 0.847,
  "errors": [],
  "summary": "Good quality (score: 0.847) with 0 error(s) detected."
}

xcomet_detect_errors

Focus on detecting and categorizing translation errors.

Parameters:

xcomet_batch_evaluate

Evaluate multiple translation pairs in a single request.

Performance Note: With the persistent server architecture (v0.3.0+), the model stays loaded in memory. Batch evaluation processes all pairs efficiently without reloading the model.