STT (whisper_worker): add VAD tuning (speech_pad_ms so soft first/last words
aren't clipped), condition_on_previous_text=False + temperature fallback +
no_speech/logprob/compression thresholds to reject noisy/quiet decodes, drop
per-segment non-speech, and a hallucination guard that blanks Whisper's classic
Korean silence/noise boilerplate ("감사합니다" 등) when no_speech_prob is high.
Barge-in (bot.mjs): stop the bot's TTS only when the speaking (green ring) stays
on for >= WSAI_BARGE_IN_MS (default 700ms), not on the first blip — cancelled if
speaking stops in time. AfterSilence 800->1000ms so trailing soft words finish.
Fragment gate (dashboard): a lone connective filler ("그러면") carries no
answerable intent -> discard as [대기] instead of replying; hidden by the noise
filter like [잡음].
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The first CUDA inference pays a large lazy cost (kernel autotune/cudnn) —
~10s for a cold TTS synth — which would blow the voice loop's ~1s budget on
the very first reply. Each worker now runs one dummy inference (TTS: a short
phrase; STT: 1s of silence) after model load and before emitting "ready", so
"ready" means "hot". Warmup failures are logged and never block startup.
Verified: first real call after startup is now TTS ~238ms / STT ~189ms
(was ~11s cold for TTS). 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Both voice backends defaulted to CPU. Fix the "CUDA unavailable" gaps so
everything that benefits from the RTX 5050 uses it:
- MeloTTS venv had CPU-only torch (2.12.0+cpu) -> installed Blackwell-capable
torch/torchaudio 2.11.0+cu128 (sm_120 verified with a real GPU matmul).
- faster-whisper CUDA loaded but transcribe() died with "libcublas.so.12 not
found": installed nvidia-cublas-cu12 + nvidia-cudnn-cu12 into the whisper
venv and inject those nvidia/*/lib dirs into the worker's LD_LIBRARY_PATH at
spawn (the loader only honours it at exec).
- WSAI_WHISPER_DEVICE / WSAI_MELO_DEVICE now default to "auto": pick CUDA when
present, else CPU, and each worker falls back to CPU if a CUDA load fails so
the voice loop never dies on a GPU-less host.
Verified end-to-end through the real backend classes: both workers report
"ready on cuda"; steady-state STT ~170ms (was ~1350ms CPU), TTS ~4s first call
vs ~23s CPU. All 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Step 3 (귀): add WhisperSTT + whisper_worker, a warm out-of-venv worker
mirroring the MeloTTS shape (whisper312 venv, small/int8 on CPU). transcribe()
closes the voice round trip (MeloTTS wav -> whisper text); utterances() turns an
injected audio_source into Utterances (Discord voice feed pending). Wired into
factory as WSAI_STT=whisper.
Also address the arbiter's TTS follow-ups:
- melo worker error handling: capture stderr (drained in a bounded background
task so the pipe can't fill), surface the real failure cause, and defend
against an empty/invalid ready line instead of dying on JSONDecodeError.
- pipeline pre-warm: load slow backends (warmup()) at startup so the first
utterance is answered warm; a warmup failure is logged, not fatal.
Verified: real TTS->STT round trip recovers the sentence near-perfectly;
warm transcribe ~1.2s (CPU). 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
