perf: run MeloTTS on the GPU (cu128 torch) + warm CUDA at startup

CPU MeloTTS serialised under concurrent load (whisper STT + bot) and blew
voice-reply TTS to 7-8s. Install the Blackwell-verified cu128 torch in the
melo venv, select the GPU via MELO_DEVICE=cuda, and do a throwaway synth at
worker startup so the one-off CUDA kernel-init (~5s) doesn't land on the
user's first reply. Measured: ~0.3s/sentence on GPU vs ~1.2-2.6s on CPU.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-14 02:22:36 +09:00
parent 44ebfeafa8
commit 927d59f805
3 changed files with 27 additions and 7 deletions

View File

@@ -66,6 +66,20 @@ def _ensure_model() -> None:
speaker_id = spk_map[LANGUAGE] if LANGUAGE in spk_map else spk_map[keys[0]]
_model = model
_speaker_id = speaker_id
# Warm the GPU once at load: the first CUDA synth pays a one-off
# kernel-init cost (~5s) that would otherwise land on the user's
# first reply. A throwaway synth here moves it to startup. No-op
# cost on CPU.
try:
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as _wt:
_wp = _wt.name
model.tts_to_file("워밍업", speaker_id, _wp, speed=SPEED)
try:
os.unlink(_wp)
except OSError:
pass
except Exception as _we: # pragma: no cover
print(f"[melo-worker] warmup synth skipped: {_we}", flush=True)
print(
f"[melo-worker] ready (lang={LANGUAGE} speed={SPEED} "
f"device={DEVICE} speakers={list(spk_map.keys())})",