NVIDIA NIM MCP Server
A production-ready Model Context Protocol (MCP) server for consuming NVIDIA NIM (NVIDIA Inference Microservices) models. Supports 50+ LLMs, multimodal models, image generation, embeddings, reranking, function calling, vision, and code-specialized models with rich metadata for intelligent agent selection.
π Features
- 10 MCP Tools: chat completion, text generation, embeddings, reranking, function calling, model listing, model info, image generation, image analysis, multimodal tasks, model comparison
- 50+ Supported Models: Llama 3.1/3.2, Nemotron 3 Ultra (550B), MiniMax M3, Kimi K2.6 (1T), DeepSeek V4 Pro, GLM 5.1, Qwen 3.5 397B, Mistral Large 3 (675B), GPT-OSS 120B, DiffusionGemma, FLUX.1, SDXL, SD3, and more
- Rich Model Metadata: licensing, hardware requirements, benchmarks, image generation specs, reasoning modes, tags for agent selection
- Advanced Filtering: by commercial use, reasoning, vision, function calling, multimodal, context length, tags, hardware
- Production-Grade: automatic retries with exponential backoff, per-minute rate limiting, structured JSON logging
- Type-Safe: full TypeScript, Zod input validation on every tool
- Docker-Ready: multi-stage Dockerfile with non-root user, health checks
- Configurable: all settings via environment variables
- Single Required Env: Only
NVIDIA_API_KEY required; all others have sensible defaults
π Prerequisites
- Node.js 18+ (for NPM installation) or Docker (for container deployment)
- A NVIDIA NGC API key (
nvapi-...)
βοΈ Installation
Option 1: NPM Global Installation (Recommended)
# Install globally
npm install -g nvidia-nim-mcp
# Run directly
nvidia-nim-mcp
Option 2: NPM Local Installation
# Initialize your project
npm init -y
# Install locally
npm install nvidia-nim-mcp
# Run with npx
npx nvidia-nim-mcp
Option 3: From Source
# Clone / download the project
cd nvidia-nim-mcp
# Install dependencies
npm install
# Build TypeScript
npm run build
Option 4: Docker
# Pull from Docker Hub (when published)
docker pull nvidia-nim-mcp
# Or build locally
docker build -t nvidia-nim-mcp .
π Configuration
Copy .env.example to .env and fill in your API key:
Only NVIDIA_API_KEY is required β all other variables have production-ready defaults:
| Variable | Required | Default | Description |
|---|
NVIDIA_API_KEY | β
| β | Your NVIDIA NGC API key |
NVIDIA_NIM_BASE_URL | β | https://integrate.api.nvidia.com/v1 | Base URL for NIM API |
DEFAULT_MODEL | β | black-forest-labs/flux.1-dev | Default model (best image generation) |
MAX_REQUESTS_PER_MINUTE | β | 40 | Rate limit cap (NVIDIA API limit) |
MAX_TOKENS_PER_REQUEST | β | 4096 | Hard cap on tokens per request |
REQUEST_TIMEOUT_MS | β | 120000 | Request timeout (ms) |
MAX_RETRIES | β | 3 | Max retry attempts on failure |
RETRY_DELAY_MS | β | 1000 | Base delay between retries (ms) |
LOG_LEVEL | β | info | error|warn|info|debug |
ENABLE_IMAGE_GENERATION | β | true | Enable image generation tools |
ENABLE_VISION | β | true | Enable vision/multimodal tools |
ENABLE_MULTIMODAL | β | true | Enable multimodal task tools |
π Running
NPM Global Installation
# Run the server
nvidia-nim-mcp
# With custom environment variables
NVIDIA_API_KEY=nvapi-your-key LOG_LEVEL=debug nvidia-nim-mcp
NPM Local Installation
# Run with npx
npx nvidia-nim-mcp
# Or add to package.json scripts
# "scripts": { "start": "nvidia-nim-mcp" }
npm start
From Source
# Development mode with auto-reload
npm run dev
# Production mode (compiled)
npm run build && npm start
Docker
# Run with environment variables
docker run --rm \
-e NVIDIA_API_KEY=nvapi-your-key \
-e LOG_LEVEL=info \
nvidia-nim-mcp
# Run in background with port mapping (if needed)
docker run -d \
--name nvidia-nim-mcp \
-e NVIDIA_API_KEY=nvapi-your-key \
nvidia-nim-mcp
Standalone Executable
# Make executable (if not already)
chmod +x dist/index.js
# Run directly
./dist/index.js
# With environment variables
NVIDIA_API_KEY=nvapi-your-key ./dist/index.js
π§ MCP Client Configuration
For Global NPM Installation
{
"mcpServers": {
"nvidia-nim": {
"command": "nvidia-nim-mcp",
"env": {
"NVIDIA_API_KEY": "nvapi-your-key-here",
"LOG_LEVEL": "info"
}
}
}
}
For Local NPM Installation
{
"mcpServers": {
"nvidia-nim": {
"command": "npx",
"args": ["nvidia-nim-mcp"],
"env": {
"NVIDIA_API_KEY": "nvapi-your-key-here",
"LOG_LEVEL": "info"
}
}
}
}
For Direct Executable Path
{
"mcpServers": {
"nvidia-nim": {
"command": "node",
"args": ["/absolute/path/to/nvidia-nim-mcp/dist/index.js"],
"env": {
"NVIDIA_API_KEY": "nvapi-your-key-here",
"LOG_LEVEL": "info"
}
}
}
}
π οΈ Available Tools
chat_completion
Multi-turn conversation with any NIM LLM.
{
"model": "nvidia/nemotron-3-ultra-550b-a55b",
"messages": [
{ "role": "user", "content": "Explain quantum computing" }
],
"temperature": 0.3,
"max_tokens": 4096
}
text_generation
Single-prompt text generation (simplified interface).
{
"prompt": "Write a haiku about machine learning",
"temperature": 0.5,
"max_tokens": 512
}
create_embeddings
Convert text(s) to vector embeddings for RAG/search.
{
"model": "nvidia/nv-embed-v1",
"input": ["NVIDIA makes GPUs", "AI runs on GPUs"],
"truncate": "END"
}
rerank_passages
Rerank passages by relevance to a query.
{
"query": "What is CUDA?",
"passages": ["CUDA is a GPU programming platform", "NIM serves AI models"],
"top_k": 3
}
function_calling
Use NIM models with tool/function calling.
{
"model": "z-ai/glm-5.1",
"messages": [{ "role": "user", "content": "What's the weather in Paris?" }],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {
"type": "object",
"properties": { "city": { "type": "string" } },
"required": ["city"]
}
}
}]
}
generate_image
Generate images from text prompts using FLUX.1, SDXL, SD3, DiffusionGemma.
{
"model": "black-forest-labs/flux.1-dev",
"prompt": "A photorealistic mountain landscape at sunset, 8K",
"width": 1024,
"height": 1024,
"steps": 30,
"cfg_scale": 3.5,
"sampler": "euler_a",
"scheduler": "simple"
}
analyze_image
Analyze and describe images using vision/multimodal models.
{
"model": "moonshotai/kimi-k2.6",
"image_url": "https://example.com/image.jpg",
"prompt": "Describe this image in detail",
"detail": "high"
}
multimodal_task
Perform multimodal tasks combining text and images.
{
"model": "minimaxai/minimax-m3",
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "Analyze this chart" },
{ "type": "image_url", "image_url": { "url": "https://example.com/chart.png" } }
]
}
],
"max_tokens": 2048
}
list_models
List available models with rich metadata and advanced filtering.
{
"category": "code",
"commercial_use": true,
"supports_reasoning": true,
"tags": ["coding", "agentic"],
"include_details": true
}
Filter Options:
category: language, embedding, reranking, vision, code, multimodal, image_generation, all
commercial_use: Filter by commercial license
supports_reasoning: Filter by reasoning capability
supports_vision: Filter by vision capability
supports_function_calling: Filter by function calling
supports_multimodal: Filter by multimodal input
min_context_length: Minimum context window (tokens)
tags: Filter by use case tags
hardware: Filter by GPU type (Hopper, Blackwell, Ampere)
include_details: Include full metadata (benchmarks, image specs, etc.)
get_model_info
Get complete metadata for a specific model.
{ "model_id": "nvidia/nemotron-3-ultra-550b-a55b" }
Returns: licensing, hardware requirements, benchmarks, image gen specs, reasoning modes, tags, supported languages, etc.
compare_models
Compare 2-5 models side-by-side across all decision factors.
{
"model_ids": [
"nvidia/nemotron-3-ultra-550b-a55b",
"deepseek-ai/deepseek-v4-pro",
"moonshotai/kimi-k2.6",
"z-ai/glm-5.1"
]
}
Returns: Structured comparison table with licensing, hardware, benchmarks, capabilities, tags, image generation specs, etc.
π¦ Supported Models (50+)
Language Models (Frontier Reasoning)