Dingo

What DataEval/dingo MCP server does

DataEval/dingo MCP server connects AI agents to Dingo, a Python tool for evaluating the quality of machine-learning datasets, LLM data, and production AI systems. The MCP interface lets an agent interact with Dingo’s evaluation functions instead of calling the library directly. It also exposes the available evaluation rules and prompts so an agent can discover the checks it can use.

Dingo supports two broad evaluation styles. Rule-based checks cover deterministic quality conditions, while LLM-based evaluations provide deeper assessment where a language model is appropriate. The underlying project describes more than 30 built-in heuristic rules, along with evaluation features for retrieval-augmented generation and hallucination detection.

How it works

The server is included in the dingo-python package and is started with Dingo’s command-line interface. It can use SSE transport on its default port, a custom SSE port, or stdio transport for desktop MCP clients. Agents communicate with the running process through MCP and can request Dingo evaluations or inspect the available rules and prompts.

Dingo’s broader evaluation workflow can read local files, PostgreSQL, MySQL, and SQLite databases, Hugging Face datasets, and S3 storage. Evaluations can apply different rules to different fields and can run locally or with Spark for large datasets.

Setup and configuration

Install the core package with:

pip install dingo-python

Optional package extras add HHEM hallucination detection, retrieval benchmarking, or all listed optional features:

pip install "dingo-python[hhem]"
pip install "dingo-python[retrieval]"
pip install "dingo-python[all]"

Start the MCP server over SSE with the default port:

dingo serve

Use dingo serve --port 9000 to select another port, or use dingo serve --transport stdio for clients such as Claude Desktop. LLM evaluation may require configuring a model API key, API URL, and model name. The example uses an OpenAI-compatible endpoint, but the material does not state that one provider is mandatory.

Tools and capabilities

DataEval/dingo MCP server supports these documented capability areas:

• Running Dingo’s rule-based evaluations.
• Running LLM-based data quality evaluations.
• Listing available rules and prompts.
• Assessing datasets and AI-generated or application data.
• Supporting RAG evaluation and hallucination detection through Dingo’s optional or built-in evaluation features.
• Producing evaluation results that can include saved bad records and field-level quality insights through Dingo’s broader execution workflow.

Limitations and notes

It also does not state that the MCP server itself provides Dingo SaaS features such as web dashboards, access control, visual reports, or REST APIs; those are described as part of the separate enterprise SaaS version.

LLM-based checks are not equivalent to local rule checks: the example configures an API key, endpoint, and model before invoking an LLM evaluator. Optional HHEM and retrieval features add separate installation requirements. Choose stdio when the client launches the process locally, and SSE when the client connects to the running server over its configured port.