# Talkies

**Category:** ☁️ Cloud Platforms  
**Repository:** https://github.com/psyb0t/docker-talkies  
**Views:** 0  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/talkies

## Description
Self-hosted MCP server for speech: ASR transcription, TTS synthesis, and file staging tools.

## Claude Desktop Quick Installation
Heuristic fallback — verify the package name and runner against the repository README before running it. Uses `npx` (confidence: low):

```json
"mcpServers": {
  "talkies": {
    "command": "npx",
    "args": ["-y","talkies"]
  }
}
```

## Documentation & README

# talkies

[![CI](https://github.com/psyb0t/docker-talkies/actions/workflows/pipeline.yml/badge.svg?branch=main)](https://github.com/psyb0t/docker-talkies/actions/workflows/pipeline.yml)
[![version](https://raw.githubusercontent.com/psyb0t/docker-talkies/badges/version.svg)](https://github.com/psyb0t/docker-talkies/releases)
[![license](https://raw.githubusercontent.com/psyb0t/docker-talkies/badges/license.svg)](LICENSE)
[![Docker Pulls](https://img.shields.io/docker/pulls/psyb0t/talkies?style=flat-square)](https://hub.docker.com/r/psyb0t/talkies)

Self-hosted speech services in one Docker image: OpenAI-compatible file
transcription and text-to-speech, Talkies live ASR over WebSocket, file
staging, model lifecycle controls, and an MCP endpoint for ASR workflows.

## Contents

- [Start here](#start-here)
- [What it provides](#what-it-provides)
- [Models at a glance](#models-at-a-glance)
  - [Reading phonemes](#reading-phonemes)
  - [Prompting Chatterbox with emotion](#prompting-chatterbox-with-emotion)
- [Documentation](#documentation)
- [Agent integrations](#agent-integrations)
- [Security in one minute](#security-in-one-minute)
- [Development](#development)

## Start here

Restrict the first boot to the models you need; otherwise the entrypoint
downloads every model in the bundled registry.

```bash
docker run --rm -it --name talkies \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/talkies-data:/data" \
  -e TALKIES_ENABLED_MODELS=whisper-large-v3-turbo,kokoro-82m \
  psyb0t/talkies:latest

curl -s http://127.0.0.1:8000/healthz
curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
  -F "file=@/path/to/clip.wav" \
  -F "model=whisper-large-v3-turbo"
```

For CUDA-only models — Parakeet-TDT, the larger Canary models, Qwen3 TTS and
Chatterbox Turbo — use `psyb0t/talkies:latest-cuda` with `--gpus all`. The
loopback port mapping keeps the service local; see
[Getting started](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/getting-started.md) for first boot and authentication.

## What it provides

| Surface | Purpose | Reference |
|---|---|---|
| `POST /v1/audio/transcriptions` | File transcription and subtitles | [HTTP API](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/api.md#transcription) |
| `WS /v1/audio/transcriptions/stream` | Live 16 kHz PCM ASR | [Streaming](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/streaming.md#live-asr-over-websocket) |
| `POST /v1/audio/speech` | Speech synthesis in six formats | [HTTP API](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/api.md#speech) |
| `GET /v1/models` | Enabled slugs and their modality | [HTTP API](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/api.md) |
| `GET /v1/audio/voices` | Per-model voice catalog with origin tags | [Models](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/models.md) |
| `GET/PUT/DELETE /v1/files/*` | Server-side file staging | [HTTP API](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/api.md#file-staging) |
| `/api/ps`, `/unload` | Model inspection and eviction | [Operations](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/operations.md#model-lifecycle) |
| `/v1/mcp` | Streamable HTTP MCP with ASR/file tools | [HTTP API](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/api.md#mcp) |
| `GET /healthz` | Liveness probe; the only unauthenticated route | [Operations](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/operations.md) |

The HTTP transcription and speech routes use the corresponding OpenAI wire
shapes where those contracts overlap. Streaming ASR, files, lifecycle controls,
and MCP are Talkies extensions.

## Models at a glance

- CPU: two Whisper models, Canary-180M-Flash, Nemotron ASR via parakeet.cpp,
  four English Sherpa-ONNX Zipformer choices, Vosk small English, two phoneme
  recognizers, and two Kokoro TTS backends.
- CUDA: the CPU set plus Parakeet-TDT, Canary 1B/Qwen ASR, five Qwen3 TTS
  variants, and Chatterbox Turbo.
- Live ASR: bundled Nemotron, Sherpa-ONNX, and Vosk are native; bundled Whisper
  is a bounded rolling decoder. Sherpa and Vosk also work through the
  OpenAI-compatible file-transcription endpoint.
- Phoneme recognition: `wav2vec2-xlsr-53-espeak` and `zipa-ipa` return the IPA
  phones that were spoken, not words, with no language model correcting them
  toward the nearest dictionary entry. Same transcription endpoint and
  timestamp options as the other ASR models; see
  [Phoneme recognition](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/models.md#phoneme-recognition).
- Per-model concurrency limits cover WebSocket, HTTP, MCP, ASR, and TTS; the
  bundled Nemotron CPU and CUDA entries admit two requests.
- Streaming TTS: Qwen3 returns incremental raw PCM for
  `response_format="pcm"`; other TTS formats and Kokoro are buffered.
- Expressive TTS: Chatterbox Turbo (English) takes 19 inline tags such as
  `[sigh]`, `[whispering]` and `[laugh]` directly in the input text. Its output
  carries a neural watermark by default; set `TALKIES_CHATTERBOX_WATERMARK` to
  false to emit unmarked audio.
- Voice cloning: drop a `.wav` into `/data/custom-voices` and it appears on
  `GET /v1/audio/voices`. Qwen3 pairs it with an optional sibling `.txt`
  transcript; Chatterbox needs only the clip, longer than five seconds.

Exact slugs, executors, tag list, and registry format:
[Models and registries](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/models.md).

### Reading phonemes

`wav2vec2-xlsr-53-espeak` and `zipa-ipa` use the same transcription call as
every other ASR slug; only the model changes. `text` comes back as a
space-separated IPA phone stream rather than words, and no language model
corrects a mispronunciation toward a real word.

```bash
curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
  -F "file=@/path/to/clip.wav" \
  -F "model=zipa-ipa"
# {"text": "a ɪ m k ə n f j u z ...", ...}
```

Add `-F "response_format=verbose_json"` (or `timestamp_granularities[]=word`)
to get each phone as a `words` entry with `start` and `end` in seconds.

### Prompting Chatterbox with emotion

Tags go inline in `input`, in square brackets, lowercase. They are real tokens
in the model's tokenizer, so only these 19 do anything — any other bracketed
word is spoken as literal text:

```
[angry] [fear] [surprised] [whispering] [advertisement] [dramatic] [narration]
[crying] [happy] [sarcastic] [clear throat] [sigh] [shush] [cough] [groan]
[sniff] [gasp] [chuckle] [laugh]
```

```bash
curl -s http://127.0.0.1:8000/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
        "model": "chatterbox-turbo",
        "voice": "builtin",
        "input": "Oh, that is hilarious. [chuckle] Anyway [sigh] back to work.",
        "response_format": "mp3"
      }' --output out.mp3
```

Swap `"voice"` for the name of any `.wav` you dropped in `/data/custom-voices`
(extension stripped) to speak the same line in a cloned voice.

## Documentation

| Guide | Contents |
|---|---|
| [Getting started](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/getting-started.md) | Run CPU/CUDA, persist data, authenticate, verify |
| [Models and registries](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/models.md) | Bundled slugs, image availability, custom registries |
| [Architecture](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/architecture.md) | Request flow, backend selection, on-disk layout |
| [HTTP API](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/api.md) | Requests, responses, files, lifecycle, MCP |
| [Streaming](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/streaming.md) | Live ASR protocol, streaming backends, PCM TTS |
| [Configuration](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/configuration.md) | Supported environment variables and limits |
| [Operations and security](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/operations.md) | Exposure, model memory, data retention, logs |
| [Development](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/development.md) | Make targets, test suites, image builds |

## Agent integrations

The [Talkies skill](https://github.com/psyb0t/docker-talkies/blob/HEAD/.agents/skills/talkies) teaches agents to use the HTTP,
WebSocket, and MCP surfaces. Install it through the shared `psyb0t` marketplace
or let Codex discover it directly from this checkout.

### Claude Code

```bash
claude plugin marketplace add psyb0t/agents
claude plugin install talkies@psyb0t
```

Claude Code prompts for the Talkies URL and, when enabled, the bearer token;
the sensitive token is stored through the client's protected configuration.

### Codex

```bash
codex plugin marketplace add psyb0t/agents
codex plugin add talkies@psyb0t
```

A marketplace install invokes the skill as `$talkies:talkies`. Codex also
discovers `.agents/skills/talkies` directly in this repository, where it is
invoked as `$talkies` without installation.

### OpenClaw

The skill and MCP bridge are published through ClawHub:

```bash
openclaw skills install @psyb0t/talkies
openclaw plugins install clawhub:@psyb0t/talkies
```

The bridge connects local stdio MCP clients to a running Talkies `/v1/mcp`
endpoint. Set `TALKIES_URL` and, when authentication is enabled,
`TALKIES_AUTH_TOKEN`.

## Security in one minute

`TALKIES_AUTH_TOKEN` enables a shared bearer token for every HTTP and WebSocket
route except `/healthz`. It is unset by default. Keep the port loopback-only or
put Talkies behind TLS, authentication, and rate limiting. If untrusted callers
can supply remote `file_path` URLs, set `TALKIES_BLOCK_PRIVATE_DOWNLOADS=true`.
See [Operations and security](https://github.com/psyb0t/docker-talkies/blob/HEAD/docs/operations.md) for the complete posture.

## Development

```bash
make check                 # lint + unit tests in the dev image
make lint                  # flake8 + mypy only
make test-unit             # fast offline unit tests
make run                   # run the CPU image locally
make test-streaming        # real CPU native WebSocket ASR test
make test-streaming-custom # real CPU Sherpa/Vosk WebSocket + HTTP tests
make test-streaming-custom-cuda # real CUDA Sherpa WebSocket + HTTP test
make compile-heavy         # regenerate the hash-locked ML requirements
make build-all             # CPU and CUDA production images
```

`make help` lists every target.

Talkies is released under the [WTFPL](https://github.com/psyb0t/docker-talkies/blob/HEAD/LICENSE). Model weights are downloaded at
runtime and have their own terms; image component notices are in
[THIRD_PARTY.md](https://github.com/psyb0t/docker-talkies/blob/HEAD/THIRD_PARTY.md). Release notes are in [CHANGELOG.md](https://github.com/psyb0t/docker-talkies/blob/HEAD/CHANGELOG.md).