🎵 Audio Sonic MCP

Turn any song into a structured "sonic signature" — extracting tempo, musical key, a 512-dimension CLAP vibe embedding, human-readable vibe tags, and a production profile — from a single local call.

Audio Sonic MCP runs entirely on your local machine (requiring no API keys, external servers, or cloud dependencies) and exposes two premium access points to the same underlying high-fidelity audio analysis engine:

🎹 Quick Taste: What You Get

1. Musician-Friendly CLI Summary (--summary mode)

🎵 SONIC SIGNATURE — my_demo.mp3  (3:24)

  TEMPO    153.8 BPM  (steady)
  KEY      G Major  ·  shifts to G Phrygian @0:30   (confidence 74%)
  VIBE     aggressive · dark · driving · hip-hop · gritty

  PRODUCTION
     Vocals     forward
     Punch      0.62  (moderate)
     Stereo     wide
     Low end    ~55 Hz dominant

  Overall confidence: 88%   ·   analyzed in 0:28 (GPU-accelerated)

2. Comprehensive JSON (Returned by MCP and CLI by default)

{
  "header": {
    "job_id": "sig_a3f9b2c1",
    "status": "success",
    "confidence_score": 0.88,
    "source_metadata": {
      "title": "Acoustic Vibe Demo",
      "duration_sec": 204,
      "source_type": "file"
    }
  },
  "sonic_signature": {
    "bpm": 153.8,
    "bpm_engine": "madmom",
    "bpm_variable": false,
    "key": "G Major",
    "key_variable": true,
    "key_map": [
      { "start_sec": 0.0,  "end_sec": 30.0, "key": "G Major" },
      { "start_sec": 30.0, "end_sec": 90.0, "key": "G Phrygian" }
    ],
    "mode_confidence": 0.74,
    "vibe_vector": [0.012, -0.034, "... 512 float dimensions ..."],
    "vibe_tags": ["aggressive", "dark", "driving", "hip-hop", "gritty"],
    "production_profile": {
      "vocal_presence": "forward",
      "transient_punch": 0.62,
      "stereo_width": "wide",
      "dominant_freq_peaks_hz": {
        "harmonic": [55.0, 110.2],
        "percussive": [125.0, 250.1]
      }
    }
  },
  "telemetry": {
    "inference_time_sec": 28.0
  }
}

⚡ Key Features

• 🥁 Tempo & Beat Tracking — Full BPM computation with variable-tempo drift detection and transient windowing.
• 🎹 Key & Harmonic Mapping — Computes structural musical key + mode, generating a detailed key_map tracking section-by-section modulations.
• 🌈 Vibe & Style Embeddings — Compiles a 512-dimensional CLAP embedding and human-readable style tags (covering energy, texture, mood, and genre) using zero-shot music vocab classification.
• 🎚️ Production Analytics — Measures vocal spatial presence, transient punch coefficients, stereo width, and dominant frequency peaks.
• 🤖 MCP-Native System — Fully exposes 4 standardized Model Context Protocol tools for instant integration into AI tools.
• 🪶 Robust Graceful Degradation — Automatically utilizes a CUDA GPU if present and falls back to CPU; gracefully degrades to HPSS and standard librosa feature arrays if heavy deep learning packages ([clap]) are omitted.
• 🔒 100% Offline & Private — All conversion, separation, and inference occur locally.

📦 Installation & Setup

System Prerequisites

Ensure you have Python 3.10+ and FFmpeg installed and accessible on your system PATH.

Installing FFmpeg:

• macOS: brew install ffmpeg
• Linux (Debian/Ubuntu): sudo apt update && sudo apt install -y ffmpeg
• Windows: Run winget install Gyan.FFmpeg via PowerShell (Administrator), or download manually from ffmpeg.org and add the bin directory to your system environment variables.

Step-by-Step Installation

1. Clone the Repository

   bash

   Copy

   git clone https://github.com/ripunjay-kashyap/audio-sonic-mcp.git
   cd audio-sonic-mcp
   

2. Initialize Virtual Environment

   bash

   Copy

   python -m venv .venv
   # Activate on macOS/Linux:
   source .venv/bin/activate
   # Activate on Windows (PowerShell):
   .venv\Scripts\activate
   

3. Install Dependencies Choose between the lightweight core engine or the full high-fidelity ML suite:

   • Option A: Full High-Fidelity ML Suite (Recommended) Includes demixing stems (Demucs) and zero-shot vibe vectors (CLAP). Requires ~4 GB disk space.
     Terminal

     Copy

     pip install -e ".[clap]"
     

   • Option B: Core Lightweight Pipeline Uses standard digital signal processing (HPSS/librosa). Rapid install and minimal footprint.
     Terminal

     Copy

     pip install -e .
     

[!NOTE] The optional [clap] stack installs torch, torchaudio, transformers, and demucs. Without these, the server automatically switches to light fallbacks (HPSS instead of Demucs, standard feature matrices instead of CLAP vectors, and leaves out vibe_tags).

🤖 MCP Client Configuration Guide

Audio Sonic MCP registers itself as a standard package script. This enables you to run it using the global executable name (audio-sonic-mcp) directly from your virtual environment's bin folder, or run the script file manually.

1. Claude Desktop Setup

Open your Claude configuration file:

• Windows: %APPDATA%\Claude\claude_desktop_config.json
• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

Add the server to your mcpServers object:

{
  "mcpServers": {
    "audio-sonic-mcp": {
      "command": "C:\\path\\to\\audio-sonic-mcp\\.venv\\Scripts\\audio-sonic-mcp.exe",
      "args": [],
      "env": {
        "JOBS_ROOT": "C:\\path\\to\\audio-sonic-mcp\\jobs"
      }
    }
  }
}

[!IMPORTANT] Windows Users: Always use double backslashes (\\) in JSON configuration paths. Point the executable directly to the .exe inside your .venv\Scripts\ directory.

2. Cursor IDE Integration

To integrate Audio Sonic MCP into Cursor's AI pane:

1. Navigate to Settings ➔ Features ➔ MCP.
2. Click + Add New MCP Server.
3. Fill in the parameters:
   • Name: audio-sonic-mcp
   • Type: command
   • Command: /path/to/audio-sonic-mcp/.venv/bin/audio-sonic-mcp (use .exe extension on Windows)

3. Windsurf Integration

Open your Windsurf MCP configurations file (typically found at ~/.codeium/windsurf/mcp_config.json) and append the configuration:

{
  "mcpServers": {
    "audio-sonic-mcp": {
      "command": "/path/to/audio-sonic-mcp/.venv/bin/python",
      "args": ["/path/to/audio-sonic-mcp/server.py"],
      "env": {
        "JOBS_ROOT": "/path/to/audio-sonic-mcp/jobs"
      }
    }
  }
}

4. Cline (VS Code Extension) Setup

Open Cline's MCP setting file (usually located at %APPDATA%\Code\User\globalStorage\saoudrizwan.claude-dev\settings\cline_mcp_settings.json or equivalent platform storage) and add:

{
  "mcpServers": {
    "audio-sonic-mcp": {
      "command": "/path/to/audio-sonic-mcp/.venv/bin/audio-sonic-mcp",
      "args": [],
      "env": {
        "JOBS_ROOT": "/path/to/audio-sonic-mcp/jobs"
      }
    }
  }
}

🤖 Interaction Flow for AI Agents & LLMs

LLMs automatically learn how to use this server by reading its exposed tool definitions. Because audio stem separation and CLAP embeddings are computationally demanding, Audio Sonic MCP uses an Asynchronous Fire-and-Forget Job Pattern.

Automated LLM Workflow

  [User Prompts LLM]
          │
          ▼
1. Submit URL ──────────────► [Tool: get_sonic_signature]
                                      │ (Returns Job ID instantly)
                                      ▼
2. Notify User ◄───────────── [LLM acknowledges job is queued]
          │
          ├───► 3. Wait 10-15s (Or proceed with other tasks)
          │
          ▼
4. Check Progress ──────────► [Tool: get_job_status]
                                      │ (Checks status: running/success/error)
                                      ▼
5. Present Signature ◄─────── [LLM formats rich output for user]

Natural Prompts to Try

• "Check the health of my audio-sonic-mcp server to make sure all ML components are ready."
• "Submit this YouTube track for sonic analysis: https://www.youtube.com/watch?v=XXXXXX."
• "Check the progress of my sonic signature job sig_a1b2c3d4 and summarize the BPM, production width, and vibe once complete."

🎚️ CLI Usage (Local Files)

For musicians, engineers, and producers working directly in the terminal, you can analyze a full-length local file directly without running any background servers:

# Get a visual, musician-friendly sonic signature digest (recommended)
python analyze_file.py "my_demo.wav" --summary

# Print full raw JSON directly to the stdout stream
python analyze_file.py "my_demo.wav"

# Dump JSON payload to a file while keeping the stdout clean
python analyze_file.py "my_demo.wav" > signature.json

CLI Command Options Reference

Supported File Formats: wav, mp3, flac, ogg, m4a, aac.

🔧 Environment Variables Reference

Configure environment options by declaring these variables in your active terminal session, container environment, or the env block of your MCP configuration file:

🐳 Docker / Podman Execution

If you prefer to avoid setting up local Python libraries, running via containers encapsulates FFmpeg, yt-dlp, and the core Python dependencies (CPU-based pipeline):

# Build the container image
docker build -t audio-sonic-mcp .

# Run the MCP server over stdio, mounting local folders for job persistence
docker run -i --rm \
  -v "$(pwd)/jobs:/app/jobs" \
  -v "$(pwd)/models:/app/models" \
  audio-sonic-mcp

To connect Claude Desktop to your Docker container, configure claude_desktop_config.json:

{
  "mcpServers": {
    "audio-sonic-mcp-docker": {
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-v", "/absolute/path/to/jobs:/app/jobs",
        "-v", "/absolute/path/to/models:/app/models",
        "audio-sonic-mcp"
      ]
    }
  }
}

⚙️ How it Works under the Hood

Audio Sonic MCP pipelines are constructed modularly, using transactional checkpoints to ensure reliability.

  LLM Agent / Claude Desktop                 Musician (Terminal)
            │                                          │
            │  MCP (stdio JSON-RPC)                    │  analyze_file.py
            ▼                                          ▼
┌──────────────────────────────────────────────────────────────────────────┐
│  Modular 6-Stage Analysis Pipeline                                       │
│                                                                          │
│  Stage 1: Ingestion   │ Pre-checks format, scans duration metadata       │
│  Stage 2: Download    │ Fetches audio tracks via yt-dlp (URLs only)      │
│  Stage 3: Conversion  │ normalizes sample formats to 44.1kHz WAV (FFmpeg)│
│  Stage 4: Separation  │ Splits stems: Vocals, Drums, Bass, Other (Demucs)│
│  Stage 5: Analysis    │ Computes BPM, modulations, key, punch (librosa)  │
│  Stage 6: Embeddings  │ Generates 512-dim zero-shot music vibe tags (CLAP)│
└─────────────────────────────────────┬────────────────────────────────────┘
                                      ▼
             Result Payload: (header · sonic_signature · telemetry)

1. Stem Demixing: Meta AI's Demucs (mdx_extra) separates the track into isolation stems (vocals, drums, bass, other). If missing, it gracefully drops back to Harmonic-Percussive Source Separation (HPSS).
2. Analysis engine: librosa extracts rhythmic and tonal structures, matching chord patterns and sub-bass movements against Krumhansl-Schmuckler and Phrygian template engines.
3. Semantic Vibe Tagging: LAION CLAP (laion/larger_clap_music_and_speech) runs zero-shot inference against high-coverage aesthetic descriptors (moods, textures, genres), choosing top candidates across stylistic poles.

🩺 Resiliency & Troubleshooting

1. One-Time Setup Download Delays

Upon the very first analysis job utilizing the full ML pipeline, demucs and transformers will download their pre-trained model weights (approximately 400 MB for Demucs, and 200 MB for CLAP).

• The server redirects download progress indicators to stderr so they do not corrupt the JSON-RPC standard stream.
• During this download, get_job_status will remain in running. Allow 1–3 minutes depending on your network speed. Subsequent startups take under 10 seconds.

2. FastMCP Concurrency Controls

Model inference on multi-staged architectures is highly CPU/VRAM intensive. To protect consumer hardware and virtual environments from crashing (OutOfMemory exceptions), Audio Sonic MCP enforces a strict global serialization lock (CONCURRENCY_LOCK).

• If you submit multiple URLs simultaneously, they will be processed sequentially.
• Polling get_job_status for subsequent jobs will report queued or running while they wait in the pipeline queue.

3. Windows Librosa Deadlock Fix

FastMCP thread dispatching under Windows can cause Numba compilation deadlocks inside background worker threads. To prevent this, Audio Sonic MCP incorporates a Pre-warming Routine (_prewarm_librosa() and _prewarm_demucs()) on launch. It forces JIT compile of resampling, HPSS, and mono-mixing functions in the main thread before starting the RPC listener.

4. BPM Accuracy and the bpm_engine Field

Tempo is estimated by madmom's RNN beat tracker. madmom is an optional dependency: it is unmaintained (latest release 0.16.1, classifiers stop at Python 3.7) and requires a Cython build, so it cannot be installed reliably everywhere and is not part of the default install.

When madmom is unavailable the pipeline falls back to librosa. That fallback is good on steady four-on-the-floor material but can lock onto a 2:3 or octave multiple of the true tempo — on one of our regression fixtures it reports 99.4 BPM against a ground truth of 148.

So the tempo is never reported unqualified. Every payload carries a bpm_engine field naming the engine that actually produced the number:

check_health reports madmom's status explicitly. To enable the accurate path:

pip install ".[beats]"

If the build fails on a recent Python, use 3.10 for the analysis environment — madmom has no wheels for newer interpreters.

5. Diagnosing with check_health

If the server reports as degraded or tools are missing, call the check_health tool or check CLI warnings. It queries:

• Availability of ffmpeg on the execution path.
• Installation status of Python packages (librosa, soundfile, mcp, etc.).
• Presence of the optional madmom beat tracker, and which bpm_engine will be used as a result.
• Access permissions to the JOBS_ROOT directory.

🛠️ Development & Testing

Run unit tests inside your virtual environment to verify the mathematical pipelines using synthesized audio waveforms:

# Install development test framework
pip install -e ".[dev]"

# Execute full suite (requires no network or model downloads)
pytest

# Test specifically CLI execution code paths
pytest tests/test_cli.py

📄 License

Distributed under the MIT License. See LICENSE for details.

© 2026 Ripunjay Kashyap. All rights reserved.