Commit Graph

2 Commits

Author SHA1 Message Date
EJClaw
51811ad251 perf(voice): run STT+TTS on the GPU by default with CPU fallback
Both voice backends defaulted to CPU. Fix the "CUDA unavailable" gaps so
everything that benefits from the RTX 5050 uses it:

- MeloTTS venv had CPU-only torch (2.12.0+cpu) -> installed Blackwell-capable
  torch/torchaudio 2.11.0+cu128 (sm_120 verified with a real GPU matmul).
- faster-whisper CUDA loaded but transcribe() died with "libcublas.so.12 not
  found": installed nvidia-cublas-cu12 + nvidia-cudnn-cu12 into the whisper
  venv and inject those nvidia/*/lib dirs into the worker's LD_LIBRARY_PATH at
  spawn (the loader only honours it at exec).
- WSAI_WHISPER_DEVICE / WSAI_MELO_DEVICE now default to "auto": pick CUDA when
  present, else CPU, and each worker falls back to CPU if a CUDA load fails so
  the voice loop never dies on a GPU-less host.

Verified end-to-end through the real backend classes: both workers report
"ready on cuda"; steady-state STT ~170ms (was ~1350ms CPU), TTS ~4s first call
vs ~23s CPU. All 12 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-18 21:02:52 +09:00
EJClaw
63fcfb7ba2 feat(stt): real Korean STT via persistent faster-whisper worker
Step 3 (귀): add WhisperSTT + whisper_worker, a warm out-of-venv worker
mirroring the MeloTTS shape (whisper312 venv, small/int8 on CPU). transcribe()
closes the voice round trip (MeloTTS wav -> whisper text); utterances() turns an
injected audio_source into Utterances (Discord voice feed pending). Wired into
factory as WSAI_STT=whisper.

Also address the arbiter's TTS follow-ups:
- melo worker error handling: capture stderr (drained in a bounded background
  task so the pipe can't fill), surface the real failure cause, and defend
  against an empty/invalid ready line instead of dying on JSONDecodeError.
- pipeline pre-warm: load slow backends (warmup()) at startup so the first
  utterance is answered warm; a warmup failure is logged, not fatal.

Verified: real TTS->STT round trip recovers the sentence near-perfectly;
warm transcribe ~1.2s (CPU). 12 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-18 18:29:32 +09:00