Make the browser search helper search the way a person does: load the
site home page, type the query into the search box one key at a time, and
press Enter — for both Google `search` and `youtube` — instead of jumping
straight to a results URL. Supports the goal of a human-like assistant.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add CHROME_USER_DATA_DIR so the browser search fallback can open Chrome
against a dedicated, Google-signed-in profile instead of a fresh anonymous
session. A returning signed-in profile is what actually avoids Google's
/sorry bot-detection page, so this is the reliable way to get browser
Google search in plain text turns. Fallback order is now CDP (broadcast
Chrome) -> persistent profile (when configured) -> ephemeral headless,
all still fail-open to the DDG/Brave/Wikipedia cascade.
Document the profile in .env.example and web_search.spec.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
browse-search.mjs only connected to the on-screen broadcast Chrome over
CDP, so browser-based Google search worked only during a live broadcast;
plain text turns fell through to the DDG cascade. Add a headless fallback
(system Chrome via channel:'chrome', else Playwright's bundled chromium)
for `search` mode so general conversation can use Google at no API cost.
`youtube` still requires the visible broadcast Chrome.
Detect Google's /sorry bot-detection interstitial structurally by URL and
fail fast so the caller fail-opens to DDG/Brave/Wikipedia instead of
treating the challenge page as empty results.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
STREAM_BROWSER becomes the broadcast *capability* master flag; the live
screen-share state (new ToolContext.broadcasting, passed per turn by the bot)
decides the backend:
- master off -> broadcast disabled, always Gemini
- master on + live on -> on-screen Chrome (visible on the stream)
- master on + live off -> Gemini
context.broadcasting is None outside the voice path (evals, text entry, older
bot) and falls back to the master flag, so current behaviour is unchanged.
This is the brain-side foundation; bot-side wiring (bridge passes broadcast
state, auto-broadcast on voice join, voice on/off toggle) follows.
Specs + docs/llm_contexts.md updated. Covered by
tests/test_web_search_broadcast_routing.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
yolo auto-approves every tool call, so a real-time search query could in
principle trigger write/shell tools. Default approval mode still auto-runs the
CLI's read-only web search in headless mode but never silently approves
destructive tools. Verified end-to-end: a grounded query returns a current
answer in ~23s with the account OAuth login. Test asserts yolo is absent;
specs and docs/llm_contexts.md updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a GEMINI_AUTH=oauth (default) sub-mode that shells out to the Gemini CLI
using the user's Google-account login instead of an API key. gemini_cli_search()
runs `gemini -p <query> -o json --skip-trust --approval-mode yolo`, strips
GEMINI_API_KEY/GOOGLE_API_KEY and sets GOOGLE_GENAI_USE_GCA=true so the CLI
selects the account OAuth method and fails fast when no login exists. Bounded by
a 30s timeout and fail-open to the DDG/Brave/Wikipedia cascade on any failure
(CLI missing, login expired, quota 429, timeout). GEMINI_AUTH=apikey keeps the
legacy REST path. Specs and docs/llm_contexts.md updated; behaviour covered by
tests/test_realtime_gemini_cli.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the two info modes in the Python brain:
- config.py: read STREAM_BROWSER / GEMINI_API_KEY / GEMINI_MODEL from env into
Settings (stream_browser, gemini_api_key, gemini_model). Verified load_settings
reads both modes.
- realtime_search.py: two fail-open backends returning the same fenced
UNTRUSTED-WEB-EXTRACT envelope: browser_search() shells the Node CDP helper to
drive the on-screen Chrome (visible on the broadcast); gemini_search() calls
the Gemini REST API with google_search grounding.
- web_search.run(): routes by mode before the DDG cascade (true->browser,
false->Gemini), falling through to DDG/Brave/Wikipedia on any miss.
- browse_and_play tool: plays a YouTube video on the shared screen (true mode
only); registered in the tool registry.
- specs + docs/llm_contexts.md updated (new Gemini LLM context); CLAUDE.md spec
registry updated.
Verified live against the running Chrome: true-mode webSearch returned real
Google results for "오늘 서울 날씨", browseAndPlay played the IU 밤편지 MV, and
false-mode degrades gracefully on a bad/absent key. A valid GEMINI_API_KEY is
still needed to confirm the real Gemini grounding output.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
