tkrmagid/javis_bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 20 Wiki Activity

feat: reuse a signed-in Chrome profile for browser web search

Browse Source 
Add CHROME_USER_DATA_DIR so the browser search fallback can open Chrome
against a dedicated, Google-signed-in profile instead of a fresh anonymous
session. A returning signed-in profile is what actually avoids Google's
/sorry bot-detection page, so this is the reliable way to get browser
Google search in plain text turns. Fallback order is now CDP (broadcast
Chrome) -> persistent profile (when configured) -> ephemeral headless,
all still fail-open to the DDG/Brave/Wikipedia cascade.

Document the profile in .env.example and web_search.spec.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-22 20:57:25 +09:00
parent 98a1825d01
commit 597207dd33
3 changed files with 86 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
.env.example
View File

@@ -98,6 +98,15 @@ CHROME_START_URL=about:blank
# on-screen browser for real-time info (search / play / read screen).
# false = no screen share; voice only, real-time info via the Gemini API.
STREAM_BROWSER=true
# Optional: profile dir for browser-based Google search in plain text turns
# (no active broadcast). When set, the search helper opens Chrome against this
# profile instead of a fresh anonymous one. Sign that profile into Google once
# (run a real Chrome with --user-data-dir=<this path> and log in) so Google
# treats later searches as a returning user and does not serve the bot-detection
# page. Leave blank to use an ephemeral headless session (works only where
# Google does not challenge it). Use a DEDICATED dir, not your everyday Chrome
# profile, to avoid the "profile in use" lock while Chrome is open.
CHROME_USER_DATA_DIR=
# Gemini auth for real-time info when STREAM_BROWSER=false.
# oauth = use the Gemini CLI with a Google-account login (no API key).
# Install once: npm i -g @google/gemini-cli ; then run `gemini` and

95
bot/scripts/stream-test/browse-search.mjs
View File

@@ -7,63 +7,86 @@
// - search : Google-search the query, return the top organic results.
// - youtube : search YouTube and play the first result.
//
// When no broadcast Chrome is reachable on CDP (e.g. a plain text conversation
// with no active broadcast), `search` falls back to a headless Chromium we
// launch ourselves, so browser-based Google search still works at zero API
// cost. `youtube` only makes sense on the visible broadcast Chrome, so it never
// uses the headless fallback.
// Backend selection for `search`:
// 1. The broadcast Chrome over CDP (visible on the Go-Live stream).
// 2. Else, if CHROME_USER_DATA_DIR is set, a persistent Chrome using that
// profile dir. Logging that dedicated profile into Google once lets Google
// treat later searches as a returning signed-in user, which avoids the
// bot-detection interstitial that blocks a fresh anonymous session.
// 3. Else a fresh ephemeral headless Chrome (works only where Google does not
// challenge the session, e.g. a non-flagged residential IP).
// `youtube` only makes sense on the visible broadcast Chrome, so it never uses
// the headless/persistent fallback.
import { chromium } from 'playwright';
const CDP = process.env.CDP_PORT || '9222';
// Use 127.0.0.1, not "localhost": in containers localhost can resolve to IPv6
// (::1) first while Chrome's CDP listens on IPv4, giving ECONNREFUSED ::1.
const CDP_HOST = process.env.CDP_HOST || '127.0.0.1';
const USER_DATA_DIR = process.env.CHROME_USER_DATA_DIR || '';
const UA =
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
'(KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36';
const query = process.argv[2] || '';
const mode = (process.argv[3] || 'search').toLowerCase();
const out = (o) => { process.stdout.write(JSON.stringify(o)); };
if (!query) { out({ ok: false, error: 'no query' }); process.exit(1); }
let browser; // playwright Browser: connected (CDP) or launched (headless)
let launched = false; // true when we launched our own headless browser
let connected; // CDP Browser (the broadcast Chrome — never kill it)
let launchedBrowser; // ephemeral headless Browser we launched
let persistent; // persistent BrowserContext we launched
let launched = false;
let page;
// Acquire a page from the broadcast Chrome over CDP, or — for search mode only —
// from a headless Chromium we launch when no broadcast Chrome is reachable.
async function acquirePage() {
// Try system Chrome (channel:'chrome') first so no extra Playwright browser
// download is needed; fall back to Playwright's bundled chromium.
async function tryLaunch(launchFn) {
let err;
for (const opts of [{ headless: true, channel: 'chrome' }, { headless: true }]) {
try {
browser = await chromium.connectOverCDP(`http://${CDP_HOST}:${CDP}`);
const ctx = browser.contexts()[0];
return await launchFn(opts);
} catch (e) {
err = e;
}
}
throw err;
}
async function acquirePage() {
// 1. Broadcast Chrome over CDP.
try {
connected = await chromium.connectOverCDP(`http://${CDP_HOST}:${CDP}`);
const ctx = connected.contexts()[0];
page = ctx.pages()[0] || (await ctx.newPage());
return;
} catch (e) {
// YouTube playback must be on the visible broadcast Chrome; no fallback.
if (mode === 'youtube') throw e;
if (mode === 'youtube') throw e; // youtube needs the visible broadcast Chrome
}
// Prefer the system Chrome the host already has (channel: 'chrome') so the
// fallback needs no extra Playwright browser download; degrade to Playwright's
// bundled chromium if no system Chrome is installed.
// 2. Persistent profile (signed-in) when configured.
if (USER_DATA_DIR) {
persistent = await tryLaunch((opts) =>
chromium.launchPersistentContext(USER_DATA_DIR, { ...opts, locale: 'ko-KR', userAgent: UA }),
);
launched = true;
let launchErr;
for (const opts of [{ headless: true, channel: 'chrome' }, { headless: true }]) {
try {
browser = await chromium.launch(opts);
break;
} catch (e) {
launchErr = e;
browser = undefined;
page = persistent.pages()[0] || (await persistent.newPage());
return;
}
}
if (!browser) throw launchErr;
const ctx = await browser.newContext({
locale: 'ko-KR',
userAgent:
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
'(KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36',
});
// 3. Ephemeral headless.
launchedBrowser = await tryLaunch((opts) => chromium.launch(opts));
launched = true;
const ctx = await launchedBrowser.newContext({ locale: 'ko-KR', userAgent: UA });
page = await ctx.newPage();
}
async function closeAll() {
try { await persistent?.close(); } catch { /* ignore */ }
try { await launchedBrowser?.close(); } catch { /* ignore */ }
try { await connected?.close(); } catch { /* ignore */ }
}
try {
await acquirePage();
page.setDefaultTimeout(20000);
@@ -86,7 +109,7 @@ try {
// and fail fast so the Python caller fail-opens to the DDG cascade instead of
// treating an empty challenge page as "no results".
if (page.url().includes('/sorry/')) {
await browser.close();
await closeAll();
out({ ok: false, error: 'google-bot-challenge', headless: launched });
process.exit(1);
}
@@ -109,9 +132,9 @@ try {
});
out({ ok: true, mode, query, count: results.length, results, headless: launched });
}
await browser.close();
await closeAll();
} catch (e) {
try { await browser?.close(); } catch { /* ignore */ }
await closeAll();
out({ ok: false, error: String(e?.message || e) });
process.exit(1);
}

27
src/jarvis/tools/builtin/web_search.spec.md
View File

@@ -24,15 +24,24 @@ path (evals, text entry) and falls back to the master flag:
`bot/scripts/stream-test/browse-search.mjs`) to Google-search the query, so
the action is visible on the Go-Live broadcast. When no broadcast Chrome is
reachable on CDP (e.g. a plain text turn with no active broadcast), the helper
falls back to launching its own headless Chrome (system `channel: 'chrome'`,
else Playwright's bundled chromium) so browser-based Google search still works
with no API cost. The `youtube` action never uses the headless fallback (it
only makes sense on the visible broadcast Chrome). Caveat: a freshly-launched
headless session with no logged-in profile can be served Google's
bot-detection interstitial (`/sorry/index`); the helper detects this
structurally by URL and fails fast, so the caller fail-opens to the DDG /
Brave / Wikipedia cascade rather than treating the challenge page as "no
results".
falls back, for `search` only, to launching its own Chrome so browser-based
Google search still works with no API cost. Fallback order:
- **CDP** (the broadcast Chrome) — preferred, visible on the stream.
- **Persistent profile** when `CHROME_USER_DATA_DIR` is set — Chrome opened
against that profile dir (system `channel: 'chrome'`, else bundled chromium).
Logging that dedicated profile into Google once lets Google treat later
searches as a returning signed-in user, which is what avoids the
bot-detection interstitial. This is the reliable way to get browser Google
search in plain text turns.
- **Ephemeral headless** otherwise — a fresh anonymous session; works only
where Google does not challenge it (e.g. a non-flagged residential IP).
The `youtube` action never uses the fallback (it only makes sense on the
visible broadcast Chrome). Caveat: an anonymous (not-signed-in) session can be
served Google's bot-detection interstitial (`/sorry/index`); the helper
detects this structurally by URL and fails fast, so the caller fail-opens to
the DDG / Brave / Wikipedia cascade rather than treating the challenge page as
"no results".
- **Gemini**: answers, with the sub-mode chosen by `cfg.gemini_auth`
(env `GEMINI_AUTH`, default `oauth`):
- `oauth` (default): `gemini_cli_search()` shells out to the Gemini CLI