perf: run MeloTTS on the GPU (cu128 torch) + warm CUDA at startup
CPU MeloTTS serialised under concurrent load (whisper STT + bot) and blew voice-reply TTS to 7-8s. Install the Blackwell-verified cu128 torch in the melo venv, select the GPU via MELO_DEVICE=cuda, and do a throwaway synth at worker startup so the one-off CUDA kernel-init (~5s) doesn't land on the user's first reply. Measured: ~0.3s/sentence on GPU vs ~1.2-2.6s on CPU. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -66,6 +66,20 @@ def _ensure_model() -> None:
|
||||
speaker_id = spk_map[LANGUAGE] if LANGUAGE in spk_map else spk_map[keys[0]]
|
||||
_model = model
|
||||
_speaker_id = speaker_id
|
||||
# Warm the GPU once at load: the first CUDA synth pays a one-off
|
||||
# kernel-init cost (~5s) that would otherwise land on the user's
|
||||
# first reply. A throwaway synth here moves it to startup. No-op
|
||||
# cost on CPU.
|
||||
try:
|
||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as _wt:
|
||||
_wp = _wt.name
|
||||
model.tts_to_file("워밍업", speaker_id, _wp, speed=SPEED)
|
||||
try:
|
||||
os.unlink(_wp)
|
||||
except OSError:
|
||||
pass
|
||||
except Exception as _we: # pragma: no cover
|
||||
print(f"[melo-worker] warmup synth skipped: {_we}", flush=True)
|
||||
print(
|
||||
f"[melo-worker] ready (lang={LANGUAGE} speed={SPEED} "
|
||||
f"device={DEVICE} speakers={list(spk_map.keys())})",
|
||||
|
||||
Reference in New Issue
Block a user