perf(voice): warm up STT+TTS workers before signalling ready

The first CUDA inference pays a large lazy cost (kernel autotune/cudnn) —
~10s for a cold TTS synth — which would blow the voice loop's ~1s budget on
the very first reply. Each worker now runs one dummy inference (TTS: a short
phrase; STT: 1s of silence) after model load and before emitting "ready", so
"ready" means "hot". Warmup failures are logged and never block startup.

Verified: first real call after startup is now TTS ~238ms / STT ~189ms
(was ~11s cold for TTS). 12 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
EJClaw
2026-08-18 21:06:23 +09:00
parent 51811ad251
commit 77d7cd8b56
2 changed files with 35 additions and 4 deletions

View File

@@ -76,8 +76,24 @@ def main() -> None:
raise
compute = _compute_for(device)
load_ms = int((time.monotonic() - t0) * 1000)
_emit({"ready": True, "ms": load_ms, "device": device, "model": model_name})
_log(f"[whisper_worker] {model_name} ready in {load_ms} ms on {device}/{compute}")
# Warm up before signalling ready: the first CUDA transcribe pays a large
# lazy cost (kernel autotune), which would slow the first real utterance.
# Run a dummy transcribe on 1s of silence here so "ready" means "hot".
warmup_ms = None
try:
import numpy as np
w = time.monotonic()
segs, _ = model.transcribe(np.zeros(16000, dtype=np.float32), language=default_lang)
for _ in segs: # segments are lazy; drain to force the actual compute
pass
warmup_ms = int((time.monotonic() - w) * 1000)
except Exception as exc:
_log(f"[whisper_worker] warmup skipped: {exc}")
_emit({"ready": True, "ms": load_ms, "device": device, "model": model_name, "warmup_ms": warmup_ms})
_log(f"[whisper_worker] {model_name} ready in {load_ms} ms on {device}/{compute} (warmup {warmup_ms} ms)")
for line in sys.stdin:
line = line.strip()