perf(voice): warm up STT+TTS workers before signalling ready

The first CUDA inference pays a large lazy cost (kernel autotune/cudnn) —
~10s for a cold TTS synth — which would blow the voice loop's ~1s budget on
the very first reply. Each worker now runs one dummy inference (TTS: a short
phrase; STT: 1s of silence) after model load and before emitting "ready", so
"ready" means "hot". Warmup failures are logged and never block startup.

Verified: first real call after startup is now TTS ~238ms / STT ~189ms
(was ~11s cold for TTS). 12 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
EJClaw
2026-08-18 21:06:23 +09:00
parent 51811ad251
commit 77d7cd8b56
2 changed files with 35 additions and 4 deletions

View File

@@ -67,8 +67,23 @@ def main() -> None:
raise
speaker_id = tts.hps.data.spk2id[lang]
load_ms = int((time.monotonic() - t0) * 1000)
_emit({"ready": True, "ms": load_ms, "device": device})
_log(f"[melo_worker] model ready in {load_ms} ms on {device}")
# Warm up before signalling ready: the first CUDA synth pays a large lazy
# cost (kernel autotune/cudnn), ~10s cold vs ~130ms hot, which would blow the
# voice loop's ~1s budget on the very first reply. Do that dummy synth here so
# "ready" means "hot". Failures must not block startup.
warmup_ms = None
try:
warm_out = os.path.expanduser("~/.cache/wsai/tts/_warmup.wav")
os.makedirs(os.path.dirname(warm_out), exist_ok=True)
w = time.monotonic()
tts.tts_to_file("워밍업", speaker_id, warm_out, speed=1.3)
warmup_ms = int((time.monotonic() - w) * 1000)
except Exception as exc:
_log(f"[melo_worker] warmup skipped: {exc}")
_emit({"ready": True, "ms": load_ms, "device": device, "warmup_ms": warmup_ms})
_log(f"[melo_worker] model ready in {load_ms} ms on {device} (warmup {warmup_ms} ms)")
for line in sys.stdin:
line = line.strip()