perf: run MeloTTS on the GPU (cu128 torch) + warm CUDA at startup
CPU MeloTTS serialised under concurrent load (whisper STT + bot) and blew voice-reply TTS to 7-8s. Install the Blackwell-verified cu128 torch in the melo venv, select the GPU via MELO_DEVICE=cuda, and do a throwaway synth at worker startup so the one-off CUDA kernel-init (~5s) doesn't land on the user's first reply. Measured: ~0.3s/sentence on GPU vs ~1.2-2.6s on CPU. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -67,6 +67,9 @@ services:
|
||||
WHISPER_MODEL: ${WHISPER_MODEL:-medium}
|
||||
WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
|
||||
WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
|
||||
# MeloTTS on the GPU (cu128 torch baked by docker/setup-melo.sh). CPU synth
|
||||
# serialised under load and pushed TTS to 7-8s; GPU does ~0.3s/sentence.
|
||||
MELO_DEVICE: ${MELO_DEVICE:-cuda}
|
||||
# Optional single-language lock for replies (empty = user's own language).
|
||||
OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-}
|
||||
# Drop the pre-loop planner LLM call to cut voice-reply latency on small
|
||||
|
||||
Reference in New Issue
Block a user