Files
javis_bot/docs/stream_browser_modes.md
javis-bot af1276cf6b refactor(search): remove the Gemini search backend
The user asked to drop it: nobody had the Gemini CLI login or a key set up,
and Exa then Wikipedia cover the voice use. webSearch is now live Chrome,
Exa, Brave (opt-in), Wikipedia. Removes the CLI/REST backends, the
GEMINI_* settings, the Docker login mount and checks, and updates the specs
and docs/llm_contexts.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 09:24:11 +09:00

3.3 KiB

Real-time info modes (STREAM_BROWSER)

The bot answers via the Python brain (bridge/server.py -> src/jarvis). Real-time info is fetched by a tool the reply engine calls.

STREAM_BROWSER is the broadcast capability master flag; the live screen-share state then decides the search backend per turn:

  • STREAM_BROWSER=true (default): broadcasting is allowed.
    • Joining a voice channel auto-starts the broadcast.
    • While the broadcast is live: drive the on-screen Chrome (CDP at CDP_PORT, default 9222) to Google-search / play YouTube / read the page — visible on the Go-Live stream (VNC display :1).
    • While the broadcast is off: the normal webSearch chain (Exa, then Brave and Wikipedia; src/jarvis/tools/builtin/web_search.spec.md), no screen needed.
    • The user can toggle the broadcast on/off by voice ("방송 켜줘" / "방송 꺼줘"), routed through an LLM tool (no hardcoded phrase matching).
  • STREAM_BROWSER=false: broadcasting is disabled entirely. Any /stream or "start broadcast" request is refused, and real-time search always uses the normal webSearch chain.

The brain receives the live broadcast state per request (ToolContext.broadcasting, threaded from the bot via the bridge); None means "no signal" (evals, text entry) and falls back to the master flag so behaviour is unchanged.

Components

Piece Path Status
Mode flag (bot) bot/src/config.ts screenBrowser, enforced in selfbot.ts done
Browser search core (Node/CDP) bot/scripts/stream-test/browse-search.mjs this change
Brain mode read src/jarvis/config.py stream_browser from env TODO
browseAndSearch tool (true) src/jarvis/tools/builtin/browse_and_search.py -> subprocess the Node core TODO
Search routing by live broadcast web_search.py uses ToolContext.broadcasting + master flag done
Live broadcast state in tool ctx tools/base.py ToolContext.broadcasting from turn_state done
Per-turn state channel reply/turn_state.py (thread-local: broadcasting in, directive out) done
Bridge passes broadcast state bridge.ts ?broadcasting= + server.py think(broadcasting=) done
Auto-broadcast on voice join (true mode) bot/src/index.ts handleJoin() starts streamer + wires hooks done
Voice broadcast on/off toggle setBroadcast tool -> broadcast_action -> bot streamer.start/stop done
/stream + start refused (false mode) handleStream() gated + setBroadcast refuses done
Specs + docs/llm_contexts.md alongside each tool done

Design decisions

  • The browser tool (Python) subprocesses a Node script rather than adding a Python CDP/playwright dependency: the Node layer already owns Chrome/CDP (broadcast-helper.mjs, selfbot.ts), so the brain shells out to node browse-search.mjs <query> and wraps the JSON result in the engine's UNTRUSTED WEB EXTRACT envelope. Keeps the 39k-line Python brain dep-free.
  • The browser path returns the same UNTRUSTED WEB EXTRACT envelope as webSearch and is fail-open: any failure returns None and the caller goes on down the webSearch chain.
  • The Gemini search backend (CLI with a Google login, or REST with a key) was removed 2026-11-18 at the user's request.