feat(bridge): gate Whisper behind Silero VAD; harden broadcast auto-start

Address review of the noise/broadcast fixes:

- STT now refuses to run Whisper on non-speech. transcribe() runs the Silero
  VAD (bundled with faster-whisper, no new dep) BEFORE the model, so noise or a
  brief loud blip with no real speech never reaches STT and can't be
  hallucinated into a transcript. The no_speech_prob/avg_logprob post-filter
  stays as a second line of defence (a clap the VAD lets through is still killed
  by Whisper's own no_speech_prob). VAD is env-tunable (VAD_THRESHOLD,
  VAD_MIN_SPEECH_MS, VAD_ENABLED) and fail-open so a VAD error never swallows a
  real utterance. Validated on real audio: synthesised Korean speech passes;
  silence, a 50ms blip and white noise are rejected.

- Broadcast auto-start no longer blocks the voice join and no longer silently
  swallows failures: wiring is synchronous, the Go-Live start runs in the
  background with a bounded retry and a loud final-failure log.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-13 14:53:54 +09:00
parent 39c7a22a12
commit 6d72e10f9c
5 changed files with 194 additions and 48 deletions

View File

@@ -34,20 +34,16 @@ export function peekStreamer(guildId: string): ScreenStreamer | undefined {
type BroadcastSession = Pick<VoiceSession, "getBroadcasting" | "onBroadcastAction">;
/**
* Wire a voice session to a streamer and auto-start the broadcast. Pure (no
* config / registry access) so it can be unit-tested with a fake streamer:
* Wire a voice session to a streamer (no auto-start). Pure (no config / registry
* access) so it can be unit-tested with a fake streamer:
* - report live state to the brain each turn,
* - let the brain toggle the stream by voice,
* - auto-start on join (skipping if already live).
*
* Auto-start failures are swallowed so a broadcast problem never blocks the
* voice conversation from coming up.
* - let the brain toggle the stream by voice.
*/
export async function coupleBroadcast(
export function wireBroadcast(
session: BroadcastSession,
streamer: ScreenStreamer,
ctx: StreamContext,
): Promise<void> {
): void {
session.getBroadcasting = () => streamer.isActive();
session.onBroadcastAction = async (action) => {
if (action === "start") {
@@ -56,16 +52,51 @@ export async function coupleBroadcast(
await streamer.stop();
}
};
try {
if (!streamer.isActive()) await streamer.start(ctx);
} catch (e) {
console.error("[broadcast] auto-start failed:", e);
}
/**
* Auto-start the broadcast, retrying a couple of times before giving up. The
* selfbot Go-Live path takes ~10-15s of humanised delays and can fail on a
* transient (login race, voice not ready yet), so a bounded retry recovers the
* common case. On final failure it logs loudly rather than silently leaving the
* user in voice with no broadcast. Never throws: a broadcast problem must not
* tear down the voice conversation. Returns whether the stream ended up live.
*/
export async function autoStartBroadcast(
streamer: ScreenStreamer,
ctx: StreamContext,
attempts = 2,
retryDelayMs = 1500,
): Promise<boolean> {
for (let attempt = 1; attempt <= attempts; attempt++) {
try {
if (streamer.isActive()) return true;
await streamer.start(ctx);
if (streamer.isActive()) return true;
console.error(
`[broadcast] auto-start attempt ${attempt}/${attempts} did not go live`,
);
} catch (e) {
console.error(`[broadcast] auto-start attempt ${attempt}/${attempts} failed:`, e);
}
if (attempt < attempts && retryDelayMs > 0) {
await new Promise((r) => setTimeout(r, retryDelayMs));
}
}
console.error(
"[broadcast] ⚠️ auto-start FAILED after retries — voice is up but NOT broadcasting " +
"(STREAM_BROWSER=true). Trigger it by voice ('방송 켜줘') or check the selfbot stream deps.",
);
return false;
}
/**
* Couple the broadcast to a freshly joined voice session when screen-share mode
* is on (STREAM_BROWSER=true). No-op in voice-only mode (STREAM_BROWSER=false).
*
* The toggle hooks are wired synchronously; the auto-start runs in the
* BACKGROUND so the ~10-15s Go-Live handshake never blocks the voice session
* from coming up. The bot can already hear and reply while the stream warms.
*/
export async function maybeCoupleBroadcast(
session: BroadcastSession,
@@ -73,7 +104,8 @@ export async function maybeCoupleBroadcast(
): Promise<void> {
if (!config.screenBrowser) return;
const streamer = await getStreamer(ctx.guildId);
await coupleBroadcast(session, streamer, ctx);
wireBroadcast(session, streamer, ctx);
void autoStartBroadcast(streamer, ctx);
}
/** Stop the broadcast for a guild if one is live (e.g. on leave). */