# Speech AI - Pronunciation, STT & TTS [Health: Active]

**Category:** 🎙️ Speech-to-Text  
**Repository:** https://github.com/fasuizu-br/speech-ai-plugin  
**GitHub Stars:** 0  
**Views:** 3  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/speech-ai-pronunciation-stt-tts

## Description
Pronunciation scoring, speech-to-text, and text-to-speech for AI agents.

## Tools
Capabilities this server exposes over MCP:

- **assess_pronunciation** — Assess English pronunciation quality from audio.

Scores pronunciation at four levels: overall, sentence, word, and phoneme.
Each score is 0-100. Phonemes are returned in both IPA and ARPAbet notation.
Sub-300ms inference latency.

Args:
    audio_base64: Base64-encoded audio data. Supports WAV, MP3, OGG, and WebM formats.
    text: The reference English text that the speaker was expected to read aloud.
    audio_format: Audio format hint — one of 'wav', 'mp3', 'ogg', 'webm'. Defaults to 'wav'.

Returns:
    dict with keys:
        - overallScore (int 0-100): Overall pronunciation quality
        - sentenceScore (int 0-100): Sentence-level fluency and accuracy
        - words (list): Per-word scores, each containing:
            - word (str): The word
            - score (int 0-100): Word pronunciation score
            - phonemes (list): Per-phoneme scores with IPA/ARPAbet notation
        - decodedTranscript (str): What the model heard (ASR transcript)
        - transcript (str): Reference text
        - confidence (float 0-1): Scoring confidence
        - warnings (list[str]): Quality warnings if any
        - audioQuality (dict): Audio metrics (SNR, peak/RMS dB, etc.)
- **check_pronunciation_service** — Check if the pronunciation assessment service is healthy and ready.

Returns:
    dict with keys:
        - status (str): 'healthy' or error state
        - modelLoaded (bool): Whether the scoring model is loaded
        - version (str): API version
- **get_phoneme_inventory** — Get the full phoneme inventory supported by the pronunciation scorer.

Returns a list of all English phonemes the engine can assess, including
ARPAbet symbol, IPA equivalent, example word, and phoneme category
(vowel, consonant, diphthong).

Returns:
    list of dicts, each with keys:
        - arpabet (str): ARPAbet symbol (e.g. 'AA', 'TH')
        - ipa (str): IPA notation
        - example (str): Example word containing the phoneme
        - category (str): vowel, consonant, or diphthong
- **transcribe_audio** — Transcribe audio to text with word-level timestamps.

Converts spoken English audio into text with optional word-level timestamps
and per-word confidence scores.

Args:
    audio_base64: Base64-encoded audio data (WAV, MP3, OGG, FLAC, WebM).
    audio_format: Audio format hint. Auto-detected from magic bytes if omitted.
    include_timestamps: Whether to include word-level timing (default: true).

Returns:
    dict with keys:
        - text (str): Full decoded transcript
        - words (list): Per-word results with timestamps, each containing:
            - word (str): The transcribed word
            - start (float): Start time in seconds
            - end (float): End time in seconds
            - confidence (float 0-1): Word-level confidence
        - audioDurationMs (int): Audio duration in milliseconds
        - metadata (dict): Processing time, audio length, model version
        - audioQuality (dict): Audio metrics (SNR, peak/RMS dB, etc.)
- **check_stt_service** — Check if the speech-to-text service is healthy and ready.

Returns:
    dict with keys:
        - status (str): 'healthy' or error state
        - modelLoaded (bool): Whether the STT model is loaded
        - version (str): API version
- **synthesize_speech** — Generate natural speech audio from English text.

Produces high-quality speech with 12 English voices.
Returns base64-encoded WAV audio (16-bit PCM, 24kHz mono) along with metadata.

Available voices:
- af_heart (default), af_bella, af_nicole, af_sarah, af_sky (American female)
- am_adam, am_michael (American male)
- bf_emma, bf_isabella (British female)
- bm_george, bm_lewis, bm_daniel (British male)

Args:
    text: English text to synthesize (1-5000 characters).
    voice: Voice ID. See list above. Defaults to 'af_heart'.
    speed: Speed multiplier from 0.5 to 2.0 (default: 1.0).

Returns:
    dict with keys:
        - audio_base64 (str): Base64-encoded WAV audio (16-bit PCM, 24kHz)
        - duration_ms (str): Audio duration in milliseconds
        - voice (str): Voice ID used
        - text_length (str): Input text character count
        - processing_ms (str): Synthesis time in milliseconds
- **list_tts_voices** — List all available text-to-speech voices with metadata.

Returns:
    dict with keys:
        - voices (list): Available voices, each with id, name, gender, accent, grade
        - defaultVoice (str): Default voice ID
- **check_tts_service** — Check if the text-to-speech service is healthy and ready.

Returns:
    dict with keys:
        - status (str): 'healthy' or error state
        - modelLoaded (bool): Whether the TTS model is loaded
        - version (str): API version
- **transcribe_audio_pro** — Transcribe audio with Whisper Large V3 Turbo — multilingual STT.

Supports 99 languages with automatic language detection, word-level
timestamps, per-word confidence scores, and optional speaker diarization
(identifies who spoke each word). Best-in-class WER (~2%).

Args:
    audio_base64: Base64-encoded audio (WAV, MP3, OGG, FLAC, WebM).
    language: Language code. Auto-detected if omitted. Supports 99 languages.
    diarize: Enable speaker diarization (default: false). When true, each word
        includes a speaker label (e.g. SPEAKER_00, SPEAKER_01).

Returns:
    dict with keys:
        - text (str): Full decoded transcript
        - words (list): Per-word results with timestamps, each containing:
            - word (str), start (float), end (float), confidence (float 0-1)
            - speaker (str|null): Speaker label when diarize=true
        - speakers (dict|null): Speaker info with count and labels
        - audioDurationMs (int): Audio duration in milliseconds
        - metadata (dict): Processing time, language, languageProbability
        - audioQuality (dict): Audio metrics (SNR, peak/RMS dB, etc.)
- **check_whisper_service** — Check if the Whisper STT Pro service is healthy and ready.

Returns:
    dict with keys:
        - status (str): 'healthy' or error state
        - modelLoaded (bool): Whether the Whisper model is loaded
        - diarizeLoaded (bool): Whether the diarization pipeline is loaded
        - version (str): API version
        - modelName (str): Whisper model name (e.g. 'large-v3-turbo')

## Claude Desktop Quick Installation
Remote MCP endpoint (confidence: high). Install path detected from listing signals. Add as a URL/SSE server in your client:

```json
"mcpServers": {
  "speech-ai-pronunciation-stt-tts": {
    "url": "https://pronunciation-mcp.thankfulfield-a7857897.eastus.azurecontainerapps.io/mcp"
  }
}
```

## Documentation

## What Speech AI - Pronunciation, STT & TTS does

Speech AI - Pronunciation, STT & TTS exposes three speech capabilities to MCP-compatible AI coding assistants: English pronunciation assessment, speech-to-text transcription, and text-to-speech generation. Pronunciation results are scored from 0 to 100 at phoneme, word, and sentence levels. Transcription includes word-level timestamps and confidence scores. Speech generation supports 12 English voices spanning American and British variants, with playback speed adjustable from 0.5x to 2.0x.

The package also defines a `speech-workflow` skill for a complete practice cycle: listen, record, receive a score, and improve. Its `pronunciation-tutor` agent supports structured interactive practice sessions.

## How it works

The integration uses the Model Context Protocol to make speech operations available to an AI assistant. A client sends requests to the hosted MCP endpoint and authenticates them with an `Ocp-Apim-Subscription-Key` header. The endpoint is:

`https://pronunciation-mcp.thankfulfield-a7857897.eastus.azurecontainerapps.io/mcp`

Available skills are `assess-pronunciation`, `transcribe-audio`, and `speech-workflow`. The first evaluates recorded audio, the second converts English audio to text with timing and confidence metadata, and the third combines the practice steps into one workflow.

## Setup and configuration

Speech AI - Pronunciation, STT & TTS can be installed through Claude Code with `/plugin install speech-ai` or added in Cursor with `/add-plugin speech-ai`. Other MCP clients can use the hosted endpoint through their MCP settings.

A manual configuration supplies the endpoint URL and an API key in the request headers:

```json
{
  "mcpServers": {
    "speech-ai": {
      "url": "https://pronunciation-mcp.thankfulfield-a7857897.eastus.azurecontainerapps.io/mcp",
      "headers": {
        "Ocp-Apim-Subscription-Key": "YOUR_API_KEY"
      }
    }
  }
}
```

Obtain an API key from brainiall.com or through Azure Marketplace. No environment variable configuration is specified in the provided installation instructions.

## Tools and capabilities

- Assess English pronunciation at phoneme, word, and sentence levels with scores from 0 to 100.
- Transcribe English audio with word-level timestamps and confidence values.
- Generate English speech with 12 American and British voices.
- Adjust generated speech speed between 0.5x and 2.0x.
- Run a structured pronunciation practice loop through the `speech-workflow` skill.
- Use the `pronunciation-tutor` agent for guided practice sessions.

## Limitations and notes

The documented speech-to-text capability is for English audio, and the documented pronunciation assessment covers English pronunciation. The API is a proprietary commercial service, while the plugin configuration files are licensed under MIT. It also describes sub-300ms transcription latency, but does not define the conditions or measurement method for that figure.

_Full upstream README: https://allmcps.com/mcp/speech-ai-pronunciation-stt-tts/readme_