Records what's now built beyond the original plan: real GPU STT/brain/TTS via
the voice-server, bracketed-emotion TTS (pitch/speed), and the dashboard's
prompt editing, bot control bar, whitelist/blacklist, 3-row turns, and log dock,
plus the dashboard<->bot control plane.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Step 3 (귀): add WhisperSTT + whisper_worker, a warm out-of-venv worker
mirroring the MeloTTS shape (whisper312 venv, small/int8 on CPU). transcribe()
closes the voice round trip (MeloTTS wav -> whisper text); utterances() turns an
injected audio_source into Utterances (Discord voice feed pending). Wired into
factory as WSAI_STT=whisper.
Also address the arbiter's TTS follow-ups:
- melo worker error handling: capture stderr (drained in a bounded background
task so the pipe can't fill), surface the real failure cause, and defend
against an empty/invalid ready line instead of dying on JSONDecodeError.
- pipeline pre-warm: load slow backends (warmup()) at startup so the first
utterance is answered warm; a warmup failure is logged, not fatal.
Verified: real TTS->STT round trip recovers the sentence near-perfectly;
warm transcribe ~1.2s (CPU). 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
--dashboard defaulted to looping mock STT forever, so the status page
piled up thousands of fake "conversations" (all mock, 0ms, same reply)
that looked like real traffic. Now it plays 3 sample utterances then
idles; a loud 데모 모드 banner states the turns are mock samples, not
real STT/Brain/TTS. Continuous demo moved behind --dashboard-loop-demo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the user-token selfbot voice path with an official Discord bot using
discord.js 14 + @discordjs/voice 0.19. The bot logs in with the stored testbot
token, joins the target voice channel, passes the DAVE/MLS E2EE handshake, and
receives per-user Opus audio via VoiceReceiver (the STT input path). ToS-safe.
Live-verified: bot joined guild "사지방" / channel "일반" and reached Ready.
Selfbot (gate.mjs/join.mjs) kept only for the deferred screenshare-video track,
which official bots cannot receive. Docs updated (README/PLAN); M1 done on bot path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
join.mjs joins the target voice channel over the proven DAVE handshake and stays
connected, mapping SPEAKING->ssrc and tallying incoming RTP (foundation for M2
audio decrypt). README records GPU=on, shared OAuth brain, natural-but-<=1s TTS,
Discord-voice STT input, and the M1..M6 milestones.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Modular async pipeline: FrameSource->Vision->context and STT/text->Brain->TTS.
All stages are Protocols; mock backends run end-to-end with no deps/keys.
Real backends included: mss screen capture, Claude vision+brain (guarded imports).
