Claude replies with markdown/backticks by default; MeloTTS's Korean text
normaliser has no entry for '`' and dies with KeyError: '`', so any reply
mentioning a command/code block crashed the whole voice turn (500 on
/api/voice-turn). Fix at the shared synth() choke point with
normalize_for_speech(), which flattens code fences/inline code/links/markdown
and guarantees no backtick reaches the worker — covering both the dashboard
voice turn and the Discord speak() bridge. Also add a PERSONA line asking the
model to avoid markdown (belt-and-suspenders; the code strip is the real fix).
errors_total never moved for turn-level failures: it was only bumped by
log("error") events, and the dashboard voice path calls turn.finish(error=...)
without logging. Emit one error-level log event from Turn.finish() when a turn
ends in error, so both the server counter and the browser SSE mirror stay
consistent, guarded to count at most once. Drop the now-redundant pipeline
log("error") to avoid double counting and remove the dead _publish stub.
Verified: raw backtick -> worker KeyError '`' reproduced; after fix real
MeloTTS synth of a backtick+fenced reply succeeds; /api/voice-turn returns 200
with a wav body on a backtick reply and errors_total stays 0, and an induced
synth failure returns 500 with errors_total incrementing to exactly 1. Full
suite 18 passed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Step 3 (귀): add WhisperSTT + whisper_worker, a warm out-of-venv worker
mirroring the MeloTTS shape (whisper312 venv, small/int8 on CPU). transcribe()
closes the voice round trip (MeloTTS wav -> whisper text); utterances() turns an
injected audio_source into Utterances (Discord voice feed pending). Wired into
factory as WSAI_STT=whisper.
Also address the arbiter's TTS follow-ups:
- melo worker error handling: capture stderr (drained in a bounded background
task so the pipe can't fill), surface the real failure cause, and defend
against an empty/invalid ready line instead of dying on JSONDecodeError.
- pipeline pre-warm: load slow backends (warmup()) at startup so the first
utterance is answered warm; a warmup failure is logged, not fatal.
Verified: real TTS->STT round trip recovers the sentence near-perfectly;
warm transcribe ~1.2s (CPU). 12 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a MeloTTS backend that runs the model in its own melo311 interpreter
as a long-lived worker (melo_worker.py), loaded once and fed synthesis
requests over a stdin/stdout JSON protocol. fd1 is split from fd2 in the
worker so MeloTTS's stdout progress chatter can't corrupt the protocol.
Each speak() writes a wav and hands the path to a pluggable sink (the
Discord voice step will swap in "play into the call"). factory wires
tts=melo; pipeline.aclose now also tears down the tts worker.
Verified (CPU): model load ~7.9s once, then a short reply synthesizes in
~0.86s (within the ~1s budget); wav is valid 44.1kHz PCM. GPU (cuda) is
selectable via WSAI_MELO_DEVICE for lower latency, pending GPU approval.
7 smoke tests still pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a stdlib-only observability site so you can open a browser and watch,
step by step: whether it is listening, what it heard, what the brain thought
and answered, how long each stage took, and whether anything errored.
- wsai/monitor.py: thread-safe telemetry hub (per-turn timed steps, status
header, error log) with a pub/sub for live push.
- wsai/dashboard.py: stdlib http.server serving a self-contained page plus an
SSE (/events) live stream; /api/state snapshot fallback.
- Pipeline emits step-by-step turn telemetry (화면 맥락 → 두뇌 → 응답) and
listening/running status; optional monitor, so existing paths are untouched.
- `python -m wsai --dashboard` starts the site (0.0.0.0:8787, WSAI_DASHBOARD_PORT)
and loops the mock voice demo so there is always live activity to watch.
- Tests cover turn recording, per-step timing, error marking, and live push.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pipeline.run() used asyncio.gather, so if one loop raised, the failing
coroutine propagated while the sibling loops kept running detached; aclose()
in the finally then closed a source/stt out from under a still-live loop.
Switch to asyncio.TaskGroup so a failing loop cancels+awaits the siblings
before teardown. Add a regression test asserting an error in the conversation
loop cancels the perception loop and still closes every source.
Make source/vision optional so the conversation loop runs with no screen
capture. Add Settings.voice() preset and `python -m wsai --voice` demo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Modular async pipeline: FrameSource->Vision->context and STT/text->Brain->TTS.
All stages are Protocols; mock backends run end-to-end with no deps/keys.
Real backends included: mss screen capture, Claude vision+brain (guarded imports).
