feat(brain): wire STREAM_BROWSER real-time modes into the reply engine (browser + Gemini)
Completes the two info modes in the Python brain: - config.py: read STREAM_BROWSER / GEMINI_API_KEY / GEMINI_MODEL from env into Settings (stream_browser, gemini_api_key, gemini_model). Verified load_settings reads both modes. - realtime_search.py: two fail-open backends returning the same fenced UNTRUSTED-WEB-EXTRACT envelope: browser_search() shells the Node CDP helper to drive the on-screen Chrome (visible on the broadcast); gemini_search() calls the Gemini REST API with google_search grounding. - web_search.run(): routes by mode before the DDG cascade (true->browser, false->Gemini), falling through to DDG/Brave/Wikipedia on any miss. - browse_and_play tool: plays a YouTube video on the shared screen (true mode only); registered in the tool registry. - specs + docs/llm_contexts.md updated (new Gemini LLM context); CLAUDE.md spec registry updated. Verified live against the running Chrome: true-mode webSearch returned real Google results for "오늘 서울 날씨", browseAndPlay played the IU 밤편지 MV, and false-mode degrades gracefully on a bad/absent key. A valid GEMINI_API_KEY is still needed to confirm the real Gemini grounding output. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
25
src/jarvis/tools/builtin/browse_and_play.spec.md
Normal file
25
src/jarvis/tools/builtin/browse_and_play.spec.md
Normal file
@@ -0,0 +1,25 @@
|
||||
## browseAndPlay Tool Spec
|
||||
|
||||
Plays a YouTube video on the shared screen so it appears on the Go-Live
|
||||
broadcast. Used when the user asks the assistant to play / watch a song, music
|
||||
video, or clip.
|
||||
|
||||
### Behaviour
|
||||
|
||||
- Public schema is a single required `query` string (what to play).
|
||||
- **Mode-gated**: only acts when `STREAM_BROWSER` is true (`cfg.stream_browser`).
|
||||
In voice-only mode (false) there is no screen to show, so it returns a short
|
||||
message and does nothing.
|
||||
- Drives the on-screen Chrome by subprocessing the Node CDP helper
|
||||
`bot/scripts/stream-test/browse-search.mjs <query> youtube`, which searches
|
||||
YouTube and plays the first result on display `:1`. The broadcast captures
|
||||
that display, so the playback is what viewers see.
|
||||
- Returns `success` with the played video's title, or a failure message if the
|
||||
helper/Chrome is unavailable. It does NOT make an LLM call.
|
||||
|
||||
### Principles
|
||||
|
||||
- The Node layer owns Chrome/CDP; the Python tool only shells out to it, so the
|
||||
brain stays free of a browser dependency.
|
||||
- Fail-open and explicit: any error returns a plain failure message rather than
|
||||
raising into the reply loop.
|
||||
Reference in New Issue
Block a user