- llm.py: log each Ollama call's caller + total/load/prompt/gen durations
so a slow voice turn is attributable to a specific internal call
(router/enrichment/digest/main); a RELOAD marker flags cold reloads.
- voice.ts: track in-flight Opus captures and abort them on session
destroy(); drop any utterance that finishes after the user left, so no
trailing post-leave VAD turns are reported.
- userbot.ts: show the speaker's Discord user ID on each transcript line
(answered and dropped) so it's clear whose audio produced the turn.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The transcript channel only showed STT and LLM seconds. Add wall-clock
start/end times and durations for listening, LLM and TTS so it's obvious
what takes long; STT surfaces as the gap between listening end and LLM start.
- bridge: emit llm_start_ms/llm_end_ms on meta and tts_*_ms on the end event
- bot: capture the listening window, assemble full timing after the stream,
and render a per-stage breakdown in the transcript message
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Proven approach: the conversation (hear+speak) runs on @discordjs/voice; the
Go-Live broadcast is a SEPARATE stream connection created on the SAME selfbot
session (exactly like a real Discord client), so ONE account hears, speaks, AND
broadcasts — no second login, no self_deaf, no voice conflict.
- voice.ts captures its own voice session_id (adapter wrap) and exposes
getSharedSession() {client, guildId, channelId, sessionId, botId}.
- broadcast.ts threads it into the StreamContext.
- selfbot.ts: when a shared session is present, build the Streamer on the
conversation client and create the stream on its session_id (no login/joinVoice/
humanPause); teardown only stops the stream (never leaves voice/destroys the
shared client). Falls back to the dedicated-account path otherwise.
Verified live: Go-Live connected in ~7s while the conversation voice stayed
ready, and the broadcast was visible in Discord — all on one account.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The transcript channel only showed successful transcripts, so dropped utterances
(the 47/50 misses) were invisible. Now every captured utterance is mirrored with
its outcome and per-stage timing:
- too-short blip (<300ms), VAD "음성 아님(VAD 차단)", "인식 실패", "답변 없음", or "ok"
- transcript + reply (or "(무응답)")
- ⏱️ stt/llm seconds
The bridge meta now carries note + stt_sec + think_sec; voice.ts fires onTurn for
every turn (not only non-empty transcripts) and for the too-short drop; userbot
formats the diagnostic line.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The /converse turn synthesised the entire reply before any audio played, so
time-to-first-audio grew with reply length. Add a streaming /converse_stream
endpoint that emits the transcript/reply first, then one audio clip per
sentence as each finishes synthesising. The Discord voice layer enqueues each
clip on arrival via the existing FIFO playQueue, so the first sentence starts
speaking while the rest are still being synthesised.
STT and the reply engine still run to completion before the first clip; only
TTS is pipelined. The non-streaming /converse and /text endpoints are
unchanged.
- bridge: language-agnostic sentence splitter (bridge/text_utils.py) + NDJSON
streaming route
- bot: ndjson() reader + converseStream() client; voice.ts plays clips
progressively
- tests: splitter unit tests + bot ndjson/converseStream tests
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the STREAM_BROWSER=true behaviour:
- handleJoin auto-starts the broadcast on voice join and wires the session to
the guild streamer; each turn reports the live state to the brain so search
routes Chrome (live) vs Gemini (off).
- New setBroadcast tool lets the user toggle the broadcast by voice ("방송
켜줘/꺼줘") via the LLM (no hardcoded phrases); it refuses when
STREAM_BROWSER=false. The directive flows brain -> bridge (broadcast_action)
-> bot streamer.start/stop, guarded by isActive() so it's idempotent.
- Per-turn IPC uses a thread-local (reply/turn_state.py) instead of threading
params through the whole engine chain: bridge sets broadcasting in, tool
records the directive out; Tool.execute exposes broadcasting on ToolContext.
Bot typecheck clean; brain covered by tests/test_set_broadcast.py (+ existing
routing tests). Specs + docs updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- voice.ts: reply playback is now a FIFO queue (AudioPlayerStatus.Idle drains
it) so concurrent speakers no longer cut each other's replies off.
- selfbot.ts: rewritten against the REAL @dank074/discord-video-stream v6 API
(verified from its d.ts): prepareStream(input, opts, signal)->{command,output},
playStream(output, streamer, {type:"go-live"}, signal), Streamer.joinVoice.
x11grab via customInputOptions; optional NVENC encode (RTX 5050) via exported
`nvenc`. package.json pinned to ^6.0.0 (was a wrong ^4.2.1).
- Dockerfile: dropped the hardcoded python3.12 LD_LIBRARY_PATH. faster-whisper
>=1.1 self-locates the pip CUDA libs; ldconfig (full path, glob) registers
them as a robust fallback. Verified: ld.so cache lists libcublas/libcudnn and
GPU whisper works with LD_LIBRARY_PATH empty.
- bridge: STT resample 48k->16k upgraded from nearest-neighbor to linear
(np.interp).
Verified: tsc clean, image builds, GPU whisper OK via ldconfig, compose valid.
