perf: run MeloTTS on the GPU (cu128 torch) + warm CUDA at startup

CPU MeloTTS serialised under concurrent load (whisper STT + bot) and blew
voice-reply TTS to 7-8s. Install the Blackwell-verified cu128 torch in the
melo venv, select the GPU via MELO_DEVICE=cuda, and do a throwaway synth at
worker startup so the one-off CUDA kernel-init (~5s) doesn't land on the
user's first reply. Measured: ~0.3s/sentence on GPU vs ~1.2-2.6s on CPU.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-14 02:22:36 +09:00
parent 44ebfeafa8
commit 927d59f805
3 changed files with 27 additions and 7 deletions

View File

@@ -67,6 +67,9 @@ services:
WHISPER_MODEL: ${WHISPER_MODEL:-medium}
WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
# MeloTTS on the GPU (cu128 torch baked by docker/setup-melo.sh). CPU synth
# serialised under load and pushed TTS to 7-8s; GPU does ~0.3s/sentence.
MELO_DEVICE: ${MELO_DEVICE:-cuda}
# Optional single-language lock for replies (empty = user's own language).
OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-}
# Drop the pre-loop planner LLM call to cut voice-reply latency on small