tkrmagid/javis_bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 20 Wiki Activity

feat(brain): add Gemini CLI OAuth path for STREAM_BROWSER=false real-time search

Browse Source 
Adds a GEMINI_AUTH=oauth (default) sub-mode that shells out to the Gemini CLI
using the user's Google-account login instead of an API key. gemini_cli_search()
runs `gemini -p <query> -o json --skip-trust --approval-mode yolo`, strips
GEMINI_API_KEY/GOOGLE_API_KEY and sets GOOGLE_GENAI_USE_GCA=true so the CLI
selects the account OAuth method and fails fast when no login exists. Bounded by
a 30s timeout and fail-open to the DDG/Brave/Wikipedia cascade on any failure
(CLI missing, login expired, quota 429, timeout). GEMINI_AUTH=apikey keeps the
legacy REST path. Specs and docs/llm_contexts.md updated; behaviour covered by
tests/test_realtime_gemini_cli.py.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-11 00:53:10 +09:00
parent 702fe8017e
commit b88def6756
8 changed files with 207 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
docs/stream_browser_modes.md
View File

@@ -6,8 +6,10 @@ info is fetched by a tool the reply engine calls. `STREAM_BROWSER` selects HOW:
- **true** (default): drive the on-screen Chrome (CDP at `CDP_PORT`, default 9222)
to Google-search / play YouTube / read the page. The action is visible on the
Go-Live broadcast. The browser is already up on the VNC display `:1`.
- **false**: use the Google Gemini API (grounded with Google Search) for
real-time info. No screen share needed (voice + API only).
- **false**: use Gemini (grounded with Google Search) for real-time info. No
screen share needed (voice + Gemini only). Auth sub-mode is `GEMINI_AUTH`:
- `oauth` (default): the Gemini CLI with a Google-account login (no API key).
- `apikey`: the REST API keyed by `GEMINI_API_KEY`.
## Components
@@ -16,9 +18,9 @@ info is fetched by a tool the reply engine calls. `STREAM_BROWSER` selects HOW:
| Mode flag (bot) | `bot/src/config.ts` `screenBrowser`, enforced in `selfbot.ts` | done |
| Browser search core (Node/CDP) | `bot/scripts/stream-test/browse-search.mjs` | this change |
| Brain mode read | `src/jarvis/config.py` `stream_browser` from env | TODO |
| Gemini key/model | `GEMINI_API_KEY`, `GEMINI_MODEL` (.env) + `config.py` | scaffolded |
| Gemini auth mode | `GEMINI_AUTH` (oauth/apikey), `GEMINI_API_KEY`, `GEMINI_MODEL` (.env) + `config.py` | done |
| `browseAndSearch` tool (true) | `src/jarvis/tools/builtin/browse_and_search.py` -> subprocess the Node core | TODO |
| `geminiSearch` tool (false) | `src/jarvis/tools/builtin/gemini_search.py` (REST, no new dep) | TODO |
| Gemini search (false) | `realtime_search.py` `gemini_cli_search()` (oauth, CLI) / `gemini_search()` (apikey, REST) | done |
| Registry (mode-gated) | `src/jarvis/tools/registry.py` `BUILTIN_TOOLS` | TODO |
| Specs + `docs/llm_contexts.md` | alongside each tool | TODO |
@@ -29,14 +31,25 @@ info is fetched by a tool the reply engine calls. `STREAM_BROWSER` selects HOW:
(`broadcast-helper.mjs`, `selfbot.ts`), so the brain shells out to
`node browse-search.mjs <query>` and wraps the JSON result in the engine's
`UNTRUSTED WEB EXTRACT` envelope. Keeps the 39k-line Python brain dep-free.
- Gemini uses the REST endpoint (`generativelanguage.googleapis.com`) via stdlib
`urllib` with the `google_search` grounding tool - no SDK dependency.
- Tools return the same `ToolExecutionResult(success, reply_text)` envelope shape
as `webSearch`, so downstream synthesis is unchanged. The brain reads
`STREAM_BROWSER` once at startup and registers the matching tool.
- Gemini has two auth sub-modes (`GEMINI_AUTH`):
- `oauth` (default): shell out to the Gemini CLI (`gemini -p <query> -o json
--skip-trust --approval-mode yolo`) authenticated by the user's Google
account login. `GEMINI_API_KEY`/`GOOGLE_API_KEY` are stripped from the child
env, and `GOOGLE_GENAI_USE_GCA=true` is set, so the CLI uses the account
login (not API-key auth) and fails fast when no login exists rather than
erroring on "no auth method". The CLI is resolved from `PATH` or
`~/.local/bin/gemini`; install with `npm i -g @google/gemini-cli` and sign
in once via interactive `gemini` ("Sign in with Google").
- `apikey`: the REST endpoint (`generativelanguage.googleapis.com`) via stdlib
`urllib` with the `google_search` grounding tool - no SDK dependency.
- Both Gemini paths and the browser path return the same
`ToolExecutionResult(success, reply_text)` envelope, and are fail-open: any
failure returns `None` and the caller degrades to the DDG/Brave/Wikipedia
cascade.
## To finish / verify
- Provide `GEMINI_API_KEY` to build + verify the false-mode path (a real call is
needed to confirm grounding output).
- Wire `config.py` + the two Python tools + registry, update specs and
`docs/llm_contexts.md` (new Gemini LLM context).
## Notes / verification
- Grounded Gemini search needs real quota. On a free tier the grounded call can
return HTTP 429 `RESOURCE_EXHAUSTED` (free-tier limit 0); the OAuth login must
be a Google account with usable Gemini quota, otherwise the path 429s and
fail-opens to DDG. The 30s subprocess timeout bounds the CLI's retry/backoff.
- Behaviour covered by `tests/test_realtime_gemini_cli.py`.