The voice loop echoed the recognised text. Wire the real brain: the Discord
voice-turn now runs STT -> ClaudeBrain.respond (with rolling conversation
history) -> TTS, so the bot actually thinks and answers. --voice-server builds
the brain by default (WSAI_BRAIN=claude, WSAI_BRAIN_MODEL overridable) and
gracefully falls back to echo if anthropic/Claude auth is unavailable. A brain
error speaks a short apology instead of killing the loop.
Verified end-to-end: an utterance wav returns X-Heard plus a distinct Claude
X-Reply and a synthesised reply wav on device=cuda. 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The bot (dave/bot.mjs) previously only joined the channel and counted audio
frames — it never fed STT or spoke back. Wire the real loop:
- Node bot: buffer each speaker's Opus->PCM utterance until AfterSilence,
wrap as WAV, POST to the Python voice-turn endpoint, then play the returned
reply wav into the channel via an AudioPlayer (ffmpeg->Opus). Skips its own
audio, dedupes overlapping subscriptions, and ignores sub-0.35s noise.
- Python: new `python -m wsai --voice-server` serves /api/voice-turn — decode
the uploaded utterance, GPU faster-whisper STT, produce a reply (echo of what
was heard for now), GPU MeloTTS synth, return the reply wav (recognised/reply
text ride along as X-Heard/X-Reply headers). Both engines pre-warmed; turns
show in the dashboard feed. MeloTTS.synth() extracted for direct wav reuse.
Echo mode verifies listening+speaking+GPU recognition entirely in Discord; the
Claude brain is the next slice. Verified the endpoint round-trip: utterance wav
-> correct Korean X-Heard/X-Reply + a WAVE reply on device=cuda. 12 tests pass,
node --check clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The status page was view-only, so there was no way to actually verify Korean
recognition end-to-end. Add a live test: record from the mic (localhost/https)
or upload an audio file (works over LAN http, where browsers block getUserMedia),
POST it to a new /api/stt endpoint that ffmpeg-normalises the blob to 16 kHz
mono and runs the real GPU faster-whisper, then shows the recognised text +
latency + device. Results also land in the live turn feed.
The dashboard now optionally holds a WhisperSTT and drives it from a private
asyncio loop thread. New `python -m wsai --stt-test` serves the page with STT
enabled and pre-warms the GPU worker so the first recognition is instant.
WhisperSTT.resolved_device is exposed for the UI.
Verified: wav and browser-style webm/opus uploads both return the correct
Korean text on device=cuda in ~240-280ms. 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
--dashboard defaulted to looping mock STT forever, so the status page
piled up thousands of fake "conversations" (all mock, 0ms, same reply)
that looked like real traffic. Now it plays 3 sample utterances then
idles; a loud 데모 모드 banner states the turns are mock samples, not
real STT/Brain/TTS. Continuous demo moved behind --dashboard-loop-demo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a stdlib-only observability site so you can open a browser and watch,
step by step: whether it is listening, what it heard, what the brain thought
and answered, how long each stage took, and whether anything errored.
- wsai/monitor.py: thread-safe telemetry hub (per-turn timed steps, status
header, error log) with a pub/sub for live push.
- wsai/dashboard.py: stdlib http.server serving a self-contained page plus an
SSE (/events) live stream; /api/state snapshot fallback.
- Pipeline emits step-by-step turn telemetry (화면 맥락 → 두뇌 → 응답) and
listening/running status; optional monitor, so existing paths are untouched.
- `python -m wsai --dashboard` starts the site (0.0.0.0:8787, WSAI_DASHBOARD_PORT)
and loops the mock voice demo so there is always live activity to watch.
- Tests cover turn recording, per-step timing, error marking, and live push.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Make source/vision optional so the conversation loop runs with no screen
capture. Add Settings.voice() preset and `python -m wsai --voice` demo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Modular async pipeline: FrameSource->Vision->context and STT/text->Brain->TTS.
All stages are Protocols; mock backends run end-to-end with no deps/keys.
Real backends included: mss screen capture, Claude vision+brain (guarded imports).
