agents/llm.md promises "play the Nth video from the top", but browseAndPlay
only ever clicked the first result. Add an optional 1-based index argument
(default 1, backward-compatible) threaded to the Node helper, which now clicks
the Nth a#video-title and clamps to the number of results returned so asking
beyond the list plays the last available video instead of failing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Make the browser search helper search the way a person does: load the
site home page, type the query into the search box one key at a time, and
press Enter — for both Google `search` and `youtube` — instead of jumping
straight to a results URL. Supports the goal of a human-like assistant.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add CHROME_USER_DATA_DIR so the browser search fallback can open Chrome
against a dedicated, Google-signed-in profile instead of a fresh anonymous
session. A returning signed-in profile is what actually avoids Google's
/sorry bot-detection page, so this is the reliable way to get browser
Google search in plain text turns. Fallback order is now CDP (broadcast
Chrome) -> persistent profile (when configured) -> ephemeral headless,
all still fail-open to the DDG/Brave/Wikipedia cascade.
Document the profile in .env.example and web_search.spec.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
browse-search.mjs only connected to the on-screen broadcast Chrome over
CDP, so browser-based Google search worked only during a live broadcast;
plain text turns fell through to the DDG cascade. Add a headless fallback
(system Chrome via channel:'chrome', else Playwright's bundled chromium)
for `search` mode so general conversation can use Google at no API cost.
`youtube` still requires the visible broadcast Chrome.
Detect Google's /sorry bot-detection interstitial structurally by URL and
fail fast so the caller fail-opens to DDG/Brave/Wikipedia instead of
treating the challenge page as empty results.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
controlBrowser search opened the results URL directly (search.naver.com?query).
Now it navigates to the homepage (www.naver.com / google.com), clicks the
on-page search box, types the query char-by-char and presses Enter — real
human-style search, visible on screen.
- control-server.mjs runs chrome-control.mjs LOCALLY on the browser host, so a
remote bot's controlBrowser (BROWSER_CONTROL_URL) drives real xdotool input
on THIS screen instead of the bot machine. Published on the LAN.
- controlBrowser tool posts to BROWSER_CONTROL_URL when set, else runs locally.
- Drop hard depends_on ollama so a browser-host doesn't start Ollama.
- gitignore .env.bak*/*.bak (a backup with tokens had been left untracked).
- Add a one-shot `search` action (site=naver/google/daum/youtube/bing) that
navigates the on-screen browser straight to the results page, so a small
model can satisfy "search X on Naver" in a single tool call instead of a
fragile navigate->type->enter chain.
- Sharpen the tool description to steer the router to controlBrowser (not
webSearch) for anything that should happen IN the visible browser.
- System prompt: answer in one short sentence (voice assistant) — also cuts
TTS time.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Install xdotool + wmctrl (late Docker layer, preserves the melo cache) so
the on-screen Chrome gets real X input (visible cursor, char-by-char typing)
instead of synthetic events; falls back to the Playwright API if absent.
- Fix active-tab detection (probe document.visibilityState instead of assuming
tab 0) so sequential ops target the right tab.
- Add back / forward / refresh; new/switch/close tabs via real keyboard
(Ctrl+T / Ctrl+<n> / Ctrl+W) when xdotool is present.
- Auto-dismiss native JS dialogs; closePopups clears blank/popup tabs.
- Report broadcast (Go-Live) state in status from the turn context.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a general browser-control tool (navigate to any site, list/open/close/
switch tabs, close popups, click, type, scroll, screenshot) for the Go-Live
Chrome, on top of the existing CDP + xdotool human-input layer (visible
cursor, char-by-char typing). Closes the gap where "open Naver" had no tool
and the model confabulated success. Also adds a system-prompt rule against
claiming actions no tool actually performed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- llm.py: log each Ollama call's caller + total/load/prompt/gen durations
so a slow voice turn is attributable to a specific internal call
(router/enrichment/digest/main); a RELOAD marker flags cold reloads.
- voice.ts: track in-flight Opus captures and abort them on session
destroy(); drop any utterance that finishes after the user left, so no
trailing post-leave VAD turns are reported.
- userbot.ts: show the speaker's Discord user ID on each transcript line
(answered and dropped) so it's clear whose audio produced the turn.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The transcript channel only showed STT and LLM seconds. Add wall-clock
start/end times and durations for listening, LLM and TTS so it's obvious
what takes long; STT surfaces as the gap between listening end and LLM start.
- bridge: emit llm_start_ms/llm_end_ms on meta and tts_*_ms on the end event
- bot: capture the listening window, assemble full timing after the stream,
and render a per-stage breakdown in the transcript message
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Proven approach: the conversation (hear+speak) runs on @discordjs/voice; the
Go-Live broadcast is a SEPARATE stream connection created on the SAME selfbot
session (exactly like a real Discord client), so ONE account hears, speaks, AND
broadcasts — no second login, no self_deaf, no voice conflict.
- voice.ts captures its own voice session_id (adapter wrap) and exposes
getSharedSession() {client, guildId, channelId, sessionId, botId}.
- broadcast.ts threads it into the StreamContext.
- selfbot.ts: when a shared session is present, build the Streamer on the
conversation client and create the stream on its session_id (no login/joinVoice/
humanPause); teardown only stops the stream (never leaves voice/destroys the
shared client). Falls back to the dedicated-account path otherwise.
Verified live: Go-Live connected in ~7s while the conversation voice stayed
ready, and the broadcast was visible in Discord — all on one account.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The transcript channel only showed successful transcripts, so dropped utterances
(the 47/50 misses) were invisible. Now every captured utterance is mirrored with
its outcome and per-stage timing:
- too-short blip (<300ms), VAD "음성 아님(VAD 차단)", "인식 실패", "답변 없음", or "ok"
- transcript + reply (or "(무응답)")
- ⏱️ stt/llm seconds
The bridge meta now carries note + stt_sec + think_sec; voice.ts fires onTurn for
every turn (not only non-empty transcripts) and for the too-short drop; userbot
formats the diagnostic line.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Set DISCORD_TRANSCRIPT_CHANNEL_ID to a text channel and the userbot posts
"들은 말: <transcript> / 답변: <reply or (무응답)>" for every voice turn, so you can
verify what the bot understood even when it doesn't answer aloud.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Critical regression: in single-account userbot mode the broadcast auto-start
logged in a SECOND session of the conversation account and called joinVoice,
which the streaming lib always sends with self_deaf:true. Voice state is
per-account, so this deafened the CONVERSATION session too and the bot silently
stopped hearing the user (STT receive broke). The Go-Live cannot connect on a
shared account anyway.
start() now refuses early (no joinVoice, no deafen) when there is no dedicated
DISCORD_STREAM_TOKEN and no normal-bot token, protecting the conversation. A
dedicated stream account re-enables the broadcast.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Confirmed root cause of "broadcast doesn't appear in Discord" in userbot mode:
the conversation and the Go-Live broadcaster were the SAME Discord account on
two sessions. Discord allows one voice presence per account, so the broadcaster's
voice connection never connects (state voiceReady:false). Proven by isolating the
broadcaster: alone it connects ("Go-Live WebRTC connected"); alongside the
conversation it times out.
The broadcaster now logs in with DISCORD_STREAM_TOKEN when set (a second burner
account dedicated to streaming), falling back to DISCORD_SELFBOT_TOKEN (correct
for normal-bot mode). When userbot mode shares one account it warns loudly with
the fix. Documents the var in .env.example.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
isActive() was a local flag set at start() time, and start() fired playStream
without awaiting the actual stream connection, so 'broadcasting' was reported
even when nothing reached Discord (auto-start log + ffmpeg/NVENC running are not
proof of transmission).
start() now waits for the streaming library's real readiness signal (the stream
connection's WebRTC reaching "connected") before declaring live. On timeout it
logs a compact connection-state diagnostic, tears the local ffmpeg pipeline down
immediately, and returns an explicit failure. isActive() reports the real live
state. Timeout is config-driven (STREAM_READY_TIMEOUT_MS, default 25s). Adds a
test for the timeout/teardown path and updates the existing leak-teardown test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Makes the Go-Live auto-start verifiable from the container logs (the reviewer
flagged that a successful broadcast left no log record), alongside the existing
loud failure log.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Address review of the noise/broadcast fixes:
- STT now refuses to run Whisper on non-speech. transcribe() runs the Silero
VAD (bundled with faster-whisper, no new dep) BEFORE the model, so noise or a
brief loud blip with no real speech never reaches STT and can't be
hallucinated into a transcript. The no_speech_prob/avg_logprob post-filter
stays as a second line of defence (a clap the VAD lets through is still killed
by Whisper's own no_speech_prob). VAD is env-tunable (VAD_THRESHOLD,
VAD_MIN_SPEECH_MS, VAD_ENABLED) and fail-open so a VAD error never swallows a
real utterance. Validated on real audio: synthesised Korean speech passes;
silence, a 50ms blip and white noise are rejected.
- Broadcast auto-start no longer blocks the voice join and no longer silently
swallows failures: wiring is synchronous, the Go-Live start runs in the
background with a bounded retry and a loud final-failure log.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "couple broadcast to voice" feature only wired auto-start into the
normal-bot join handler. Userbot mode (the only mode that can actually Go
Live) was added later with Go-Live deferred to a "stage 2" that never
landed, so the running deployment had no path to start a broadcast.
Extract the coupling into a shared bot/src/broadcast.ts (auto-start on
join, report live state to the brain each turn, voice toggle) and wire it
into both index.ts and userbot.ts so both modes behave identically. Add a
behavioural test for the auto-start + toggle contract.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The /converse turn synthesised the entire reply before any audio played, so
time-to-first-audio grew with reply length. Add a streaming /converse_stream
endpoint that emits the transcript/reply first, then one audio clip per
sentence as each finishes synthesising. The Discord voice layer enqueues each
clip on arrival via the existing FIFO playQueue, so the first sentence starts
speaking while the rest are still being synthesised.
STT and the reply engine still run to completion before the first clip; only
TTS is pipelined. The non-streaming /converse and /text endpoints are
unchanged.
- bridge: language-agnostic sentence splitter (bridge/text_utils.py) + NDJSON
streaming route
- bot: ndjson() reader + converseStream() client; voice.ts plays clips
progressively
- tests: splitter unit tests + bot ndjson/converseStream tests
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The bot, bridge and melo worker boot together, but the MeloTTS model takes
tens of seconds to load. If the bot logged in and auto-joined the voice channel
before the voice was warm, the first reply synthesised to nothing and was
silently dropped.
- bridge /health now reports `tts_ready`. For MeloTTS this pings the worker,
which only binds its HTTP port AFTER the model is loaded (main() warms before
serve_forever()), so a successful ping is a precise "voice is warm" signal.
- The bot polls /health and waits for `tts_ready` before logging in. It does
not wait on brain_ready (the reply engine / Whisper load lazily on the first
turn — a slow first turn is fine, a silent one is the bug). After a 180s cap
it proceeds anyway so a TTS load failure degrades to text-only.
Live-verified: startup logs show "⏳ MeloTTS 준비 대기 중" then
"✓ 음성(MeloTTS) 준비 완료 — 로그인 진행" then "✓ 유저봇 로그인", in that order.
The broadcast subtitle rule treated any ko* track as Korean, so YouTube's
auto-generated (자동 생성) Korean track would switch captions on. Match only
human-authored tracks (kind !== 'asr', vss_id not 'a.*') and pass the full
track object to setOption so YouTube selects the manual track rather than the
same-languageCode ASR one. Captions stay off when only an auto track exists.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The selfbot library's native voice connection cannot complete Discord's current
DAVE/v8 voice handshake and times out (15s). Root cause: DAVE E2EE (mandated
~March 2026). The bundled @discordjs/voice 0.18.0 also lacked DAVE.
Fix: upgrade @discordjs/voice to 0.19.2 + add @snazzah/davey (the DAVE protocol
lib) and drive the selfbot client through @discordjs/voice's joinVoiceChannel /
receiver via the client's voiceAdapterCreator — i.e. reuse the normal-bot
VoiceSession with a user-account channel. Verified: the userbot now reaches a
READY voice connection (host test) and joins the channel in-container with no
timeout. This also fixes the normal-bot voice path under DAVE. Drops the
hand-rolled selfbot-native receiver (audio.ts).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A user account is the only kind Discord lets Go Live, so add a userbot run mode:
when DISCORD_BOT_TOKEN is absent but DISCORD_SELFBOT_TOKEN is set, the app runs
as a userbot instead of the normal slash-command bot.
Stage 1 (conversation): log in with discord.js-selfbot-v13, auto-join
DISCORD_VOICE_CHANNEL_ID on startup (or "!자비스 join" when unset), listen via the
selfbot VoiceReceiver (pcm), transcribe+reply through the bridge, and speak the
reply back with playAudio. Shared PCM/WAV helpers extracted to audio.ts.
run-bot.sh now starts in either userbot or normal mode. Go-Live broadcast +
broadcast-coupled routing land in stage 2 on the same voice connection.
ToS note: selfbots violate Discord ToS; burner account only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses review findings on the dockerized stack:
- Container Chrome search was dead: add --remote-debugging-port + a non-default
--user-data-dir (Chrome 136+ refuses CDP on the default profile), add the
playwright dep (browse-search.mjs connectOverCDP) with browser download
skipped, and connect to 127.0.0.1 not "localhost" (container localhost -> ::1
while Chrome binds IPv4). Verified: browse-search returns real results.
- Broadcast toggle reliability: always offer setBroadcast in screen-share mode
(the embedding/keyword router dropped it for non-English utterances) and make
its description force a tool call. "방송 꺼줘"->stop now 5/5; no false triggers.
- Stop the broadcast on voice leave (no orphaned stream).
- Security: bind VNC/noVNC to loopback by default (VNC_BIND override) and the
bridge to the container loopback (BRIDGE_HOST=127.0.0.1), not published.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the STREAM_BROWSER=true behaviour:
- handleJoin auto-starts the broadcast on voice join and wires the session to
the guild streamer; each turn reports the live state to the brain so search
routes Chrome (live) vs Gemini (off).
- New setBroadcast tool lets the user toggle the broadcast by voice ("방송
켜줘/꺼줘") via the LLM (no hardcoded phrases); it refuses when
STREAM_BROWSER=false. The directive flows brain -> bridge (broadcast_action)
-> bot streamer.start/stop, guarded by isActive() so it's idempotent.
- Per-turn IPC uses a thread-local (reply/turn_state.py) instead of threading
params through the whole engine chain: bridge sets broadcasting in, tool
records the directive out; Tool.execute exposes broadcasting on ToolContext.
Bot typecheck clean; brain covered by tests/test_set_broadcast.py (+ existing
routing tests). Specs + docs updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
First increment of the STREAM_BROWSER real-time-info modes (true = browser,
false = Gemini):
- browse-search.mjs: drives the on-screen Chrome via CDP so the action shows on
the broadcast. `search` returns the top Google results (title/url/snippet);
`youtube` plays the first result. Verified live: real-time Seoul weather
results, and IU 'Good Day' MV playback.
- .env.example: GEMINI_API_KEY / GEMINI_MODEL for the false-mode Gemini account.
- docs/stream_browser_modes.md: architecture + integration map (brain config,
the two mode-gated tools, registry, design decisions) for the remaining wiring.
The Python brain wiring (config.py mode/gemini fields, browseAndSearch +
geminiSearch tools, registry, specs, llm_contexts) lands next - it needs a
running brain and a Gemini key to verify, rather than committing untested edits
into the 39k-line engine.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses review of the STREAM_BROWSER / broadcast-defaults work:
- SelfbotStreamer now spawns broadcast-helper.mjs on stream start and kills it on
stop/self-end (alongside capture + keepalive). The ad-skip, subtitle rule and
fullscreen-toolbar-hide are therefore guaranteed broadcast-wide defaults tied
to the broadcast - not a manual process. Fail-open: if node/Chrome deps are
absent the stream runs without the helper. Verified the helper is a child of
the broadcast holder and armed.
- Enforce STREAM_BROWSER at the streamer (start() returns early when
screenBrowser===false), so EVERY caller including stream-hold.ts is voice-only
when it's off, not just the slash command. stream-hold.ts reads STREAM_BROWSER.
- Fix broadcast-helper fullscreen: resolve the window of the tab actually in
HTML5 fullscreen (via its CDP targetId) instead of the first HTTP tab, so the
right Chrome window is toggled when multiple windows exist.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add STREAM_BROWSER (.env) gating screen-share/browser mode. false => the
/자비스 stream command stays voice + API/MCP only (no Go-Live); true (default)
=> screen share as before. (Browser-driven info retrieval in true mode is a
follow-up build; the bot has no browser-control tools yet.)
- Make the two test-time fixes broadcast-wide defaults via broadcast-helper.mjs:
it now also watches every tab for HTML5 fullscreen and toggles Chrome window
fullscreen so the address bar is hidden for ANY video (xfwm4 won't hide it on
'f' alone), restoring on exit. Subtitles were already enforced per video.
scenario.mjs drops its own fullscreen toggle and relies on the helper.
- Revert the test-settings env vars from .env.example (not wanted).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses review of the ad/subtitle work (the ad-skip.mjs -> broadcast-helper.mjs
rename's other half; the prior commit only recorded the deletion):
- ad mute leak: the ad-skipper muted during an ad but never un-muted, so the
main video stayed silent after the first ad. Save the pre-ad muted/playbackRate
and restore them when the ad ends (verified: muted false -> true -> false).
- captions were only applied once when scenario.mjs ran, not for the whole
broadcast. The persistent helper now applies the rule (OFF by default, Korean
ON if offered) per video and ENFORCES it every tick - one-shot did not hold
because YouTube silently re-enabled captions (verified it stays off across 8s).
- ad-skip + captions merged into broadcast-helper.mjs (one CDP process).
- the 60fps MV test now lives in the repo: scenario.mjs gains MV_QUERY (search +
auto-pick the first >=60fps result) and WATCH_SECONDS, plus the
fullscreen-toolbar-hide fix. The broadcast runs via the committed
stream-hold.ts (audio + keepalive), not an out-of-repo copy.
- document the test env vars (CDP_PORT, HOLD_MS, TEST_*, MV_QUERY, WATCH_SECONDS).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses review of the ad/subtitle work:
- ad mute leak: the ad-skipper muted during an ad but never un-muted, so the
main video stayed silent after the first ad. Save the pre-ad muted/playbackRate
and restore them when the ad ends (verified: muted false -> true -> false).
- captions were only applied once when scenario.mjs ran, not for the whole
broadcast. Move the rule (OFF by default, Korean ON if offered) into the
persistent helper so it runs per video, and ENFORCE it every tick - one-shot
did not hold because YouTube silently re-enabled captions (verified it now
stays off across 8s).
- merge ad-skip.mjs + captions into broadcast-helper.mjs (one CDP process).
- the actual 60fps MV test now lives in the repo: scenario.mjs gains MV_QUERY
(search + auto-pick the first >=60fps result) and WATCH_SECONDS, with the
fullscreen-toolbar-hide fix. The broadcast runs via the committed
stream-hold.ts (audio + keepalive), not an out-of-repo copy.
- document the test env vars (CDP_PORT, HOLD_MS, TEST_*, MV_QUERY, WATCH_SECONDS)
in .env.example.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds ad-skip.mjs: connects over CDP and injects a watcher into every tab
(current and future) that clicks "Skip ad" the moment it appears, closes overlay
ads, and fast-forwards unskippable ads (seek-to-end + 16x + mute) so they clear
in ~1s. Self-contained (no extension, no hosts/network changes) and reconnects
across Chrome restarts. Documented in the README.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two broadcast-experience improvements:
- Audio: the Go-Live stream was video-only. Capture the desktop sound (the
default PipeWire/Pulse sink monitor, @DEFAULT_MONITOR@) as a second ffmpeg
input and mux AAC into the mpegts; the library re-encodes it to Opus for
Discord. Controlled by STREAM_AUDIO / STREAM_AUDIO_SOURCE (default on). ffmpeg
inherits XDG_RUNTIME_DIR to reach the pulse socket. Verified: the streamer now
reports "Found audio stream" and the monitor carries Chrome audio (~-11 dB).
- Subtitles: in the browse scenario, default captions OFF, but auto-enable a
Korean track when the video offers one (getOption captions tracklist ->
setOption / unloadModule).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
On the streamed VNC desktop (xfwm4), Chrome did not hide its toolbar when a
video entered HTML5 fullscreen via 'f' - the window was full-screen (outerHeight
1080) but the tab/address bar stayed, leaving only 988px of content, so the
address bar bled into the Go-Live broadcast.
Toggle Chrome-initiated browser fullscreen via CDP (Browser.setWindowBounds
windowState fullscreen) around the 'f' step. That reliably hides the toolbar
(innerHeight 1080 vs 988); the toolbar is restored on exit, so normal browsing
still shows it. Verified live: clean full-screen video, no toolbar.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Go-Live broadcast looked badly choppy: video and scrolling stuttered while
the cursor stayed smooth. Root cause is TigerVNC: it only refreshes its
framebuffer while a VNC client is attached, but the broadcast reads that
framebuffer with x11grab (not as a VNC client). With no viewer attached the
captured screen idled at ~1.5 fps (measured 3/30 distinct frames); the cursor
looked smooth only because x11grab overlays the live cursor on every frame.
- Add a headless RFB keepalive (vnc-keepalive.ts) that stays connected for the
life of the stream and requests incremental framebuffer updates at the stream
framerate. SelfbotStreamer starts it on broadcast start and tears it down on
stop/self-end. Measured 3/30 -> 57/60 distinct frames at 60 fps. Fail-open;
authenticates with VNC_PASSWORD or the ~/.config/tigervnc/passwd file.
- Fix a resource leak: when the Go-Live ended on its own, only the active flag
was cleared, leaving the x11grab->nvenc ffmpeg running forever (pinning a CPU
core while no media was transmitted, with only the gateway TCP left and no UDP
media). The self-end path now tears down capture, keepalive and voice like
stop() does.
- Tests for both paths (self-end teardown; keepalive DES auth, port mapping,
password resolution). Add @types/bun so bun:test typechecks; document the
keepalive and recommended Chrome flags in README and .env.example.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
bringIntoView returned the last boundingBox() unconditionally after the
scroll loop exhausted, so an element still outside the viewport would be
clicked anyway. Validate the final box against the actual viewport bounds
on both axes (innerWidth/innerHeight) and return null otherwise, so
humanClick fails instead of clicking an off-screen coordinate.
Address review accuracy: humanClick used DOM scrollIntoViewIfNeeded and fell
back to Playwright locator.click() when an element had no box - neither is real
input. Now it brings elements into view with a real wheel scroll and throws if
there is no on-screen box (no synthetic click). Header comment and README
corrected: xdotool injects synthetic X input (not a physical HID device), and
all actions are real input while the CDP/DOM API is used only to read state.
Make every action real keyboard/mouse via xdotool, not just the visible
browsing: address-bar navigation (Ctrl+L + char-by-char typing), the YouTube
settings gear -> 화질 -> 1080p menu (real clicks, verified hd1080), the autoplay
toggle, the play button, and fullscreen via the real 'f' key (F11 isn't honored
by this WM; 'f' yields true 1080p fullscreen without pausing). CDP/DOM API is
now used only to read state for verification.
The startup catch cleared this.active unconditionally. In a stop()+restart
race during the slow login/pauses, the first attempt's catch would fire after
the second start() had already taken the lock, unlocking it mid-startup and
letting a third start() race in. Guard the active/state reset with
`this.controller === controller`, matching the field-null and playStream
.finally guards.
Verified live: stop during login then restart keeps the restart's lock
(active stays true), and it clears to false only once truly stopped; no crash.
The human-pause delays leave start() in-flight for several seconds, which
exposed two races:
- stop() during a pause only ended the pause; start() continued and called
joinVoice on the streamer stop() had already nulled (null deref).
- `active` was set only just before go-live, so a second /stream during the
delay passed the guard and both calls raced on the same overwritten streamer.
Now start() locks `active` before any await, keeps controller/streamer/capture
as local refs, and calls signal.throwIfAborted() after each await so an
interleaved stop() unwinds into a catch that tears down via the local refs and
clears instance state only if it still points at this attempt. isActive() now
reflects "starting" during the delay too.
Verified live: concurrent start is rejected ("이미 송출 중입니다"), stop() mid-
startup returns a cancel message with isActive=false and no uncaught error, and
the happy path still goes live and tears down cleanly. tsc --noEmit passes.
Joining voice and starting the broadcast instantly looks like a bot. Add
randomised, human-plausible pauses (~0.9-2.2s after coming online before
joining the channel, ~2.5-5s after joining before hitting Go Live) so the
cadence isn't machine-instant or fingerprintable. The pause resolves
immediately on stop() so teardown never hangs mid-wait.
Verified live: end-to-end join -> settle -> Go Live took ~8s before the
stream went live, held for 15s, and tore down cleanly. tsc --noEmit passes.
Address review: the capture ffmpeg had no -b:v, so it encoded at nvenc's
low default (~2.47 Mbps) and the library then re-encoded to 8 Mbps, which
only upscaled already-lost detail. The double encode also kept CPU decode
+ scale + re-encode in the library, contradicting the "GPU handles it"
claim.
Now the system ffmpeg produces the final Discord-ready H264 in one pass
(-b:v/-maxrate at the configured bitrate, -bf 0, 1s keyframes, yuv420p,
-forced-idr) and prepareStream uses noTranscoding:true to remux only. One
GPU encode, no library decode/scale/re-encode.
Verified locally: high-motion source fills 8.7 Mbps at these args (vs the
~2.47 Mbps no-bitrate default), real :1 desktop holds 60fps at realtime,
and the capture -> copy/remux chain yields h264 1920x1080 yuv420p 60fps
has_b_frames=0. tsc --noEmit passes. Live Discord test pending reboot.
Bump the default broadcast to 1080p 60fps at 8 Mbps and route both encode
stages through the GPU (RTX 5050, h264_nvenc) so 60fps stays smooth without
loading the 4-core host.
- selfbot.ts: capture ffmpeg uses h264_nvenc when streamHw is on (falls back
to software x264 otherwise), and prepareStream now passes Encoders.nvenc()
so the library's transcode runs on the GPU too. Guard loadLib for Encoders.
- config.ts: VNC_FRAMERATE default 30 -> 60, VNC_BITRATE_KBPS 4000 -> 8000.
- .env.example: document the new 1080p60/8 Mbps defaults and STREAM_HW.
Verified locally: h264_nvenc x11grab holds a steady 60fps with headroom,
Encoders.nvenc() returns valid h264_nvenc settings, and tsc --noEmit passes.
Live Discord voice-channel verification pending a host reboot.
End-to-end verified with a real burner token + voice channel: login OK, posts
to the text channel, joins voice, and Go-Live streams the host :1 desktop.
- selfbot.ts now captures the X display with the SYSTEM ffmpeg (reliable
x11grab) and pipes it into prepareStream, instead of relying on the lib's
bundled libav input devices (not portable). Capture process is killed on stop.
- package.json: trustedDependencies (node-av, @lng2004/node-datachannel) so the
native streaming deps build automatically on bun install (incl. Docker).
- Dropped the unused nvenc path (the lib's exported `nvenc` is undefined at
runtime); software H264 encode for now.
get-token.ts now writes the Remote Auth URL as a 512x512 QR image
(/tmp/javis_qr.png, override via QR_OUT) in addition to printing the link, so
it can be sent to the user and scanned from a second screen with the Discord
mobile app. Adds the qrcode dependency.
bot/src/get-token.ts uses discord.js-selfbot-v13 DiscordAuthWebsocket: it
prints the Discord Remote Auth URL (https://discord.com/ra/<code> — the same
thing a login QR encodes). Open it on a phone with the Discord app, approve the
"New login" prompt, and the user token is written to .env as
DISCORD_SELFBOT_TOKEN. Works from a single mobile device (no second screen, no
password, no browser devtools). `bun run token`.
