Commit Graph

5 Commits

Author SHA1 Message Date
javis-bot
4176a68873 fix(selfbot): smooth VNC capture via keepalive + stop ffmpeg leak on stream end
The Go-Live broadcast looked badly choppy: video and scrolling stuttered while
the cursor stayed smooth. Root cause is TigerVNC: it only refreshes its
framebuffer while a VNC client is attached, but the broadcast reads that
framebuffer with x11grab (not as a VNC client). With no viewer attached the
captured screen idled at ~1.5 fps (measured 3/30 distinct frames); the cursor
looked smooth only because x11grab overlays the live cursor on every frame.

- Add a headless RFB keepalive (vnc-keepalive.ts) that stays connected for the
  life of the stream and requests incremental framebuffer updates at the stream
  framerate. SelfbotStreamer starts it on broadcast start and tears it down on
  stop/self-end. Measured 3/30 -> 57/60 distinct frames at 60 fps. Fail-open;
  authenticates with VNC_PASSWORD or the ~/.config/tigervnc/passwd file.
- Fix a resource leak: when the Go-Live ended on its own, only the active flag
  was cleared, leaving the x11grab->nvenc ffmpeg running forever (pinning a CPU
  core while no media was transmitted, with only the gateway TCP left and no UDP
  media). The self-end path now tears down capture, keepalive and voice like
  stop() does.
- Tests for both paths (self-end teardown; keepalive DES auth, port mapping,
  password resolution). Add @types/bun so bun:test typechecks; document the
  keepalive and recommended Chrome flags in README and .env.example.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 15:21:44 +09:00
javis-bot
5137fdeaf7 selfbot streaming: verified live; capture via system ffmpeg x11grab
Some checks failed
Release / build-windows (push) Blocked by required conditions
Release / build-macos (arm64, macos-latest) (push) Blocked by required conditions
Release / build-macos (x64, macos-15-intel) (push) Blocked by required conditions
Release / release-main (push) Blocked by required conditions
Release / release-develop (push) Blocked by required conditions
Release / semantic-release (push) Successful in 24s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 10m1s
Release / build-linux (push) Failing after 7m35s
End-to-end verified with a real burner token + voice channel: login OK, posts
to the text channel, joins voice, and Go-Live streams the host :1 desktop.

- selfbot.ts now captures the X display with the SYSTEM ffmpeg (reliable
  x11grab) and pipes it into prepareStream, instead of relying on the lib's
  bundled libav input devices (not portable). Capture process is killed on stop.
- package.json: trustedDependencies (node-av, @lng2004/node-datachannel) so the
  native streaming deps build automatically on bun install (incl. Docker).
- Dropped the unused nvenc path (the lib's exported `nvenc` is undefined at
  runtime); software H264 encode for now.
2026-06-10 10:38:28 +09:00
javis-bot
7aac92fc2c token helper: render auth link as a scannable QR PNG
Some checks failed
Release / semantic-release (push) Successful in 26s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 9m54s
Release / build-linux (push) Failing after 7m13s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
get-token.ts now writes the Remote Auth URL as a 512x512 QR image
(/tmp/javis_qr.png, override via QR_OUT) in addition to printing the link, so
it can be sent to the user and scanned from a second screen with the Discord
mobile app. Adds the qrcode dependency.
2026-06-09 21:03:31 +09:00
javis-bot
b56c9c7721 Address remaining review items (queue, selfbot v6 API, ldconfig, resample)
Some checks failed
Release / semantic-release (push) Successful in 22s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 9m56s
Release / build-linux (push) Failing after 7m15s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
- voice.ts: reply playback is now a FIFO queue (AudioPlayerStatus.Idle drains
  it) so concurrent speakers no longer cut each other's replies off.
- selfbot.ts: rewritten against the REAL @dank074/discord-video-stream v6 API
  (verified from its d.ts): prepareStream(input, opts, signal)->{command,output},
  playStream(output, streamer, {type:"go-live"}, signal), Streamer.joinVoice.
  x11grab via customInputOptions; optional NVENC encode (RTX 5050) via exported
  `nvenc`. package.json pinned to ^6.0.0 (was a wrong ^4.2.1).
- Dockerfile: dropped the hardcoded python3.12 LD_LIBRARY_PATH. faster-whisper
  >=1.1 self-locates the pip CUDA libs; ldconfig (full path, glob) registers
  them as a robust fallback. Verified: ld.so cache lists libcublas/libcudnn and
  GPU whisper works with LD_LIBRARY_PATH empty.
- bridge: STT resample 48k->16k upgraded from nearest-neighbor to linear
  (np.interp).

Verified: tsc clean, image builds, GPU whisper OK via ldconfig, compose valid.
2026-06-09 18:47:25 +09:00
javis-bot
c4abf63f38 Add Discord-native hybrid front-end for Jarvis (bot + bridge)
Some checks failed
Release / semantic-release (push) Successful in 59s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 13m45s
Release / build-linux (push) Failing after 7m47s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
Transform isair/jarvis into a Discord-controlled voice assistant running on
the Ubuntu VNC desktop, keeping the mature ~39k-line Python brain intact.

- bot/ (Node + bun, discord.js): /자비스 slash commands (ephemeral),
  voice channel join + voice receive/playback, pluggable VNC screen broadcast
  (selfbot live / noVNC / screenshot)
- bridge/ (Python, Flask): wraps jarvis STT + run_reply_engine + Piper TTS
  behind a thin localhost HTTP API
- .env.example, scripts/ (start_bridge/start_bot/dev), README rewrite,
  docs/language-comparison.md and docs/vnc-xfce-setup.md

Language decision: hybrid (Python brain + Node/bun Discord layer) because
Discord blocks bot video; native screen broadcast only works via a Node
selfbot library.
2026-06-09 14:51:05 +09:00