Commit Graph

5 Commits

Author SHA1 Message Date
javis-bot
ef6f6ff57d feat(stream): STREAM_BROWSER flag + make toolbar-hide/subtitles broadcast-wide
- Add STREAM_BROWSER (.env) gating screen-share/browser mode. false => the
  /자비스 stream command stays voice + API/MCP only (no Go-Live); true (default)
  => screen share as before. (Browser-driven info retrieval in true mode is a
  follow-up build; the bot has no browser-control tools yet.)
- Make the two test-time fixes broadcast-wide defaults via broadcast-helper.mjs:
  it now also watches every tab for HTML5 fullscreen and toggles Chrome window
  fullscreen so the address bar is hidden for ANY video (xfwm4 won't hide it on
  'f' alone), restoring on exit. Subtitles were already enforced per video.
  scenario.mjs drops its own fullscreen toggle and relies on the helper.
- Revert the test-settings env vars from .env.example (not wanted).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 16:17:29 +09:00
javis-bot
208fbbc851 feat(selfbot): broadcast desktop audio + smart subtitles in the browse scenario
Two broadcast-experience improvements:

- Audio: the Go-Live stream was video-only. Capture the desktop sound (the
  default PipeWire/Pulse sink monitor, @DEFAULT_MONITOR@) as a second ffmpeg
  input and mux AAC into the mpegts; the library re-encodes it to Opus for
  Discord. Controlled by STREAM_AUDIO / STREAM_AUDIO_SOURCE (default on). ffmpeg
  inherits XDG_RUNTIME_DIR to reach the pulse socket. Verified: the streamer now
  reports "Found audio stream" and the monitor carries Chrome audio (~-11 dB).
- Subtitles: in the browse scenario, default captions OFF, but auto-enable a
  Korean track when the video offers one (getOption captions tracklist ->
  setOption / unloadModule).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 15:50:32 +09:00
javis-bot
ad0caa8142 feat: 1080p60 NVENC selfbot broadcast (8 Mbps default)
Bump the default broadcast to 1080p 60fps at 8 Mbps and route both encode
stages through the GPU (RTX 5050, h264_nvenc) so 60fps stays smooth without
loading the 4-core host.

- selfbot.ts: capture ffmpeg uses h264_nvenc when streamHw is on (falls back
  to software x264 otherwise), and prepareStream now passes Encoders.nvenc()
  so the library's transcode runs on the GPU too. Guard loadLib for Encoders.
- config.ts: VNC_FRAMERATE default 30 -> 60, VNC_BITRATE_KBPS 4000 -> 8000.
- .env.example: document the new 1080p60/8 Mbps defaults and STREAM_HW.

Verified locally: h264_nvenc x11grab holds a steady 60fps with headroom,
Encoders.nvenc() returns valid h264_nvenc settings, and tsc --noEmit passes.
Live Discord voice-channel verification pending a host reboot.
2026-06-10 11:17:44 +09:00
javis-bot
b56c9c7721 Address remaining review items (queue, selfbot v6 API, ldconfig, resample)
Some checks failed
Release / semantic-release (push) Successful in 22s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 9m56s
Release / build-linux (push) Failing after 7m15s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
- voice.ts: reply playback is now a FIFO queue (AudioPlayerStatus.Idle drains
  it) so concurrent speakers no longer cut each other's replies off.
- selfbot.ts: rewritten against the REAL @dank074/discord-video-stream v6 API
  (verified from its d.ts): prepareStream(input, opts, signal)->{command,output},
  playStream(output, streamer, {type:"go-live"}, signal), Streamer.joinVoice.
  x11grab via customInputOptions; optional NVENC encode (RTX 5050) via exported
  `nvenc`. package.json pinned to ^6.0.0 (was a wrong ^4.2.1).
- Dockerfile: dropped the hardcoded python3.12 LD_LIBRARY_PATH. faster-whisper
  >=1.1 self-locates the pip CUDA libs; ldconfig (full path, glob) registers
  them as a robust fallback. Verified: ld.so cache lists libcublas/libcudnn and
  GPU whisper works with LD_LIBRARY_PATH empty.
- bridge: STT resample 48k->16k upgraded from nearest-neighbor to linear
  (np.interp).

Verified: tsc clean, image builds, GPU whisper OK via ldconfig, compose valid.
2026-06-09 18:47:25 +09:00
javis-bot
c4abf63f38 Add Discord-native hybrid front-end for Jarvis (bot + bridge)
Some checks failed
Release / semantic-release (push) Successful in 59s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 13m45s
Release / build-linux (push) Failing after 7m47s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
Transform isair/jarvis into a Discord-controlled voice assistant running on
the Ubuntu VNC desktop, keeping the mature ~39k-line Python brain intact.

- bot/ (Node + bun, discord.js): /자비스 slash commands (ephemeral),
  voice channel join + voice receive/playback, pluggable VNC screen broadcast
  (selfbot live / noVNC / screenshot)
- bridge/ (Python, Flask): wraps jarvis STT + run_reply_engine + Piper TTS
  behind a thin localhost HTTP API
- .env.example, scripts/ (start_bridge/start_bot/dev), README rewrite,
  docs/language-comparison.md and docs/vnc-xfce-setup.md

Language decision: hybrid (Python brain + Node/bun Discord layer) because
Discord blocks bot video; native screen broadcast only works via a Node
selfbot library.
2026-06-09 14:51:05 +09:00