Commit Graph

7 Commits

Author SHA1 Message Date
javis-bot
568a1ae50b fix(bot): auto-start broadcast on voice join in userbot mode
The "couple broadcast to voice" feature only wired auto-start into the
normal-bot join handler. Userbot mode (the only mode that can actually Go
Live) was added later with Go-Live deferred to a "stage 2" that never
landed, so the running deployment had no path to start a broadcast.

Extract the coupling into a shared bot/src/broadcast.ts (auto-start on
join, report live state to the brain each turn, voice toggle) and wire it
into both index.ts and userbot.ts so both modes behave identically. Add a
behavioural test for the auto-start + toggle contract.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-13 14:39:29 +09:00
javis-bot
932aacef6e feat(bot): delay login until MeloTTS voice is warm
The bot, bridge and melo worker boot together, but the MeloTTS model takes
tens of seconds to load. If the bot logged in and auto-joined the voice channel
before the voice was warm, the first reply synthesised to nothing and was
silently dropped.

- bridge /health now reports `tts_ready`. For MeloTTS this pings the worker,
  which only binds its HTTP port AFTER the model is loaded (main() warms before
  serve_forever()), so a successful ping is a precise "voice is warm" signal.
- The bot polls /health and waits for `tts_ready` before logging in. It does
  not wait on brain_ready (the reply engine / Whisper load lazily on the first
  turn — a slow first turn is fine, a silent one is the bug). After a 180s cap
  it proceeds anyway so a TTS load failure degrades to text-only.

Live-verified: startup logs show " MeloTTS 준비 대기 중" then
"✓ 음성(MeloTTS) 준비 완료 — 로그인 진행" then "✓ 유저봇 로그인", in that order.
2026-06-12 21:59:50 +09:00
javis-bot
921a757371 feat(bot): userbot (selfbot) mode — voice conversation on one user account (stage 1)
A user account is the only kind Discord lets Go Live, so add a userbot run mode:
when DISCORD_BOT_TOKEN is absent but DISCORD_SELFBOT_TOKEN is set, the app runs
as a userbot instead of the normal slash-command bot.

Stage 1 (conversation): log in with discord.js-selfbot-v13, auto-join
DISCORD_VOICE_CHANNEL_ID on startup (or "!자비스 join" when unset), listen via the
selfbot VoiceReceiver (pcm), transcribe+reply through the bridge, and speak the
reply back with playAudio. Shared PCM/WAV helpers extracted to audio.ts.
run-bot.sh now starts in either userbot or normal mode. Go-Live broadcast +
broadcast-coupled routing land in stage 2 on the same voice connection.

ToS note: selfbots violate Discord ToS; burner account only.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-11 11:08:39 +09:00
javis-bot
03375a77d0 fix(docker+brain): make container Chrome search work, harden, improve toggle
Addresses review findings on the dockerized stack:
- Container Chrome search was dead: add --remote-debugging-port + a non-default
  --user-data-dir (Chrome 136+ refuses CDP on the default profile), add the
  playwright dep (browse-search.mjs connectOverCDP) with browser download
  skipped, and connect to 127.0.0.1 not "localhost" (container localhost -> ::1
  while Chrome binds IPv4). Verified: browse-search returns real results.
- Broadcast toggle reliability: always offer setBroadcast in screen-share mode
  (the embedding/keyword router dropped it for non-English utterances) and make
  its description force a tool call. "방송 꺼줘"->stop now 5/5; no false triggers.
- Stop the broadcast on voice leave (no orphaned stream).
- Security: bind VNC/noVNC to loopback by default (VNC_BIND override) and the
  bridge to the container loopback (BRIDGE_HOST=127.0.0.1), not published.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-11 10:44:56 +09:00
javis-bot
ca86390407 feat: couple broadcast to voice + voice-controlled broadcast toggle
Completes the STREAM_BROWSER=true behaviour:
- handleJoin auto-starts the broadcast on voice join and wires the session to
  the guild streamer; each turn reports the live state to the brain so search
  routes Chrome (live) vs Gemini (off).
- New setBroadcast tool lets the user toggle the broadcast by voice ("방송
  켜줘/꺼줘") via the LLM (no hardcoded phrases); it refuses when
  STREAM_BROWSER=false. The directive flows brain -> bridge (broadcast_action)
  -> bot streamer.start/stop, guarded by isActive() so it's idempotent.
- Per-turn IPC uses a thread-local (reply/turn_state.py) instead of threading
  params through the whole engine chain: bridge sets broadcasting in, tool
  records the directive out; Tool.execute exposes broadcasting on ToolContext.

Bot typecheck clean; brain covered by tests/test_set_broadcast.py (+ existing
routing tests). Specs + docs updated.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-11 10:08:30 +09:00
javis-bot
ef6f6ff57d feat(stream): STREAM_BROWSER flag + make toolbar-hide/subtitles broadcast-wide
- Add STREAM_BROWSER (.env) gating screen-share/browser mode. false => the
  /자비스 stream command stays voice + API/MCP only (no Go-Live); true (default)
  => screen share as before. (Browser-driven info retrieval in true mode is a
  follow-up build; the bot has no browser-control tools yet.)
- Make the two test-time fixes broadcast-wide defaults via broadcast-helper.mjs:
  it now also watches every tab for HTML5 fullscreen and toggles Chrome window
  fullscreen so the address bar is hidden for ANY video (xfwm4 won't hide it on
  'f' alone), restoring on exit. Subtitles were already enforced per video.
  scenario.mjs drops its own fullscreen toggle and relies on the helper.
- Revert the test-settings env vars from .env.example (not wanted).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 16:17:29 +09:00
javis-bot
c4abf63f38 Add Discord-native hybrid front-end for Jarvis (bot + bridge)
Some checks failed
Release / semantic-release (push) Successful in 59s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 13m45s
Release / build-linux (push) Failing after 7m47s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
Transform isair/jarvis into a Discord-controlled voice assistant running on
the Ubuntu VNC desktop, keeping the mature ~39k-line Python brain intact.

- bot/ (Node + bun, discord.js): /자비스 slash commands (ephemeral),
  voice channel join + voice receive/playback, pluggable VNC screen broadcast
  (selfbot live / noVNC / screenshot)
- bridge/ (Python, Flask): wraps jarvis STT + run_reply_engine + Piper TTS
  behind a thin localhost HTTP API
- .env.example, scripts/ (start_bridge/start_bot/dev), README rewrite,
  docs/language-comparison.md and docs/vnc-xfce-setup.md

Language decision: hybrid (Python brain + Node/bun Discord layer) because
Discord blocks bot video; native screen broadcast only works via a Node
selfbot library.
2026-06-09 14:51:05 +09:00