Compare commits
7 Commits
5b6a67963a
...
v1.2.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1efabe03b1 | ||
|
|
09cd4c5e31 | ||
|
|
00ce813845 | ||
|
|
c56ce1eb30 | ||
|
|
597207dd33 | ||
|
|
98a1825d01 | ||
|
|
da27c5a306 |
33
.env.example
33
.env.example
@@ -98,12 +98,28 @@ CHROME_START_URL=about:blank
|
||||
# on-screen browser for real-time info (search / play / read screen).
|
||||
# false = no screen share; voice only, real-time info via the Gemini API.
|
||||
STREAM_BROWSER=true
|
||||
# Optional: profile dir for browser-based Google search in plain text turns
|
||||
# (no active broadcast). When set, the search helper opens Chrome against this
|
||||
# profile instead of a fresh anonymous one. Sign that profile into Google once
|
||||
# (run a real Chrome with --user-data-dir=<this path> and log in) so Google
|
||||
# treats later searches as a returning user and does not serve the bot-detection
|
||||
# page. Leave blank to use an ephemeral headless session (works only where
|
||||
# Google does not challenge it). Use a DEDICATED dir, not your everyday Chrome
|
||||
# profile, to avoid the "profile in use" lock while Chrome is open.
|
||||
CHROME_USER_DATA_DIR=
|
||||
# Gemini auth for real-time info when STREAM_BROWSER=false.
|
||||
# oauth = use the Gemini CLI with a Google-account login (no API key).
|
||||
# Install once: npm i -g @google/gemini-cli ; then run `gemini` and
|
||||
# "Sign in with Google". Uses the CLI's built-in web-search grounding.
|
||||
# apikey = legacy REST path; needs GEMINI_API_KEY below
|
||||
# (get one at https://aistudio.google.com/app/apikey).
|
||||
# NOTE (2026-06): Google is blocking personal Google accounts on this
|
||||
# path ("This client is no longer supported for Gemini Code Assist for
|
||||
# individuals"). Workspace/org accounts may still work; personal
|
||||
# accounts should use apikey below instead.
|
||||
# apikey = REST path; needs GEMINI_API_KEY below
|
||||
# (get one at https://aistudio.google.com/app/apikey). Recommended for
|
||||
# personal Google accounts now that individual OAuth login is blocked.
|
||||
# Either way, real-time search fail-opens to DDG/Brave/Wikipedia if Gemini is
|
||||
# unavailable, so this is optional, not required.
|
||||
GEMINI_AUTH=oauth
|
||||
GEMINI_API_KEY=
|
||||
GEMINI_MODEL=gemini-2.0-flash
|
||||
@@ -174,11 +190,18 @@ VOICE_SILENCE_MS=800
|
||||
JARVIS_ROLE=full
|
||||
|
||||
# --- GPU per OS: pick the matching compose override via COMPOSE_FILE ---
|
||||
# Ubuntu (nvidia-container-toolkit / CDI):
|
||||
# IMPORTANT: the file separator is OS-specific. Linux/macOS use ":" (colon);
|
||||
# Windows uses ";" (semicolon), because ":" is taken by the drive letter (C:).
|
||||
# Using the wrong one makes Docker treat the whole string as a single missing
|
||||
# filename ("...gpu-windows.yml: The system cannot find the file specified").
|
||||
# Ubuntu / macOS (nvidia-container-toolkit / CDI):
|
||||
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
|
||||
# Windows 11 (Docker Desktop + WSL2 + NVIDIA):
|
||||
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml
|
||||
# Windows 11 (Docker Desktop + WSL2 + NVIDIA) — note the ";" separator:
|
||||
# COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml
|
||||
# Browser-only host (no GPU needed): leave COMPOSE_FILE unset (base only).
|
||||
# Default below is the Linux form; Windows users must change ":" to ";" AND
|
||||
# swap gpu-linux for gpu-windows. If unsure, comment this out and pass the
|
||||
# files explicitly: docker compose -f docker-compose.yml -f <gpu-override> ...
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
|
||||
|
||||
# --- Browser HOST (JARVIS_ROLE=browser) — e.g. this LAN machine ---
|
||||
|
||||
6
.gitattributes
vendored
6
.gitattributes
vendored
@@ -7,3 +7,9 @@
|
||||
|
||||
# PowerShell is more forgiving but the same logic applies.
|
||||
*.ps1 text eol=crlf
|
||||
|
||||
# Shell scripts run inside the Linux container; they MUST stay LF even when
|
||||
# checked out on Windows. autocrlf=true would otherwise inject CR and break
|
||||
# `set -o pipefail`, shebangs, and heredocs (e.g. docker/setup-melo.sh failing
|
||||
# the image build with "set: pipefail: invalid option name").
|
||||
*.sh text eol=lf
|
||||
|
||||
11
Dockerfile
11
Dockerfile
@@ -63,7 +63,11 @@ RUN ls -d /opt/venv/lib/python*/site-packages/nvidia/cublas/lib \
|
||||
# Heavy layer (torch CPU + transformers + MeCab); placed before the app
|
||||
# COPY so it stays cached across source-only changes. ---
|
||||
COPY docker/setup-melo.sh /app/docker/setup-melo.sh
|
||||
RUN bash /app/docker/setup-melo.sh
|
||||
# Strip CR before running: a Windows checkout (autocrlf) yields CRLF, which makes
|
||||
# bash read line 18 as `set -euxo pipefail\r` and abort with
|
||||
# "set: pipefail: invalid option name". .gitattributes pins *.sh to LF, but this
|
||||
# keeps the build working even on a not-yet-renormalised working tree.
|
||||
RUN sed -i 's/\r$//' /app/docker/setup-melo.sh && bash /app/docker/setup-melo.sh
|
||||
|
||||
# --- Human input + window management for the on-screen Chrome control tool.
|
||||
# Placed AFTER the heavy melo layer so it doesn't bust that cache. xdotool
|
||||
@@ -81,6 +85,11 @@ RUN cd /app/bot && bun install --frozen-lockfile || bun install
|
||||
COPY . /app
|
||||
WORKDIR /app
|
||||
|
||||
# Normalise all container shell scripts to LF. On a Windows checkout (autocrlf)
|
||||
# these arrive as CRLF, which would break their shebangs at runtime (entrypoint,
|
||||
# run-*.sh) the same way it broke setup-melo.sh at build time.
|
||||
RUN find /app/docker /app/scripts -name '*.sh' -exec sed -i 's/\r$//' {} +
|
||||
|
||||
# --- Default Piper voice (best-effort at build; entrypoint retries if absent) ---
|
||||
RUN bash docker/download-piper.sh || true
|
||||
|
||||
|
||||
@@ -76,12 +76,14 @@ docker compose up -d --build
|
||||
매번 `-f`를 치기 싫으면 `.env`에 한 줄 넣어두면 그냥 `docker compose up -d`로 됩니다(override가 자동 적용):
|
||||
|
||||
```bash
|
||||
# Linux
|
||||
# Linux / macOS (구분자 = 콜론 ":")
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
|
||||
# Windows 11
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml
|
||||
# Windows 11 (구분자 = 세미콜론 ";" — 콜론은 드라이브 문자 C: 와 충돌)
|
||||
COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml
|
||||
```
|
||||
|
||||
> ⚠️ `COMPOSE_FILE`의 파일 구분자는 OS마다 다릅니다: Linux/macOS는 `:`, Windows는 `;`. Windows에서 `:`를 쓰면 Docker가 전체를 파일 하나 이름으로 읽어 `... The system cannot find the file specified` 에러가 납니다. 헷갈리면 `COMPOSE_FILE`을 비워두고 실행 시 직접 지정하세요: `docker compose -f docker-compose.yml -f docker-compose.gpu-windows.yml up -d --build`.
|
||||
|
||||
> Linux와 Windows는 GPU를 컨테이너에 넣는 방식이 달라서 override 파일이 갈립니다. Linux는 CDI(`devices: nvidia.com/gpu=all`), Windows(Docker Desktop)는 Compose의 `deploy.resources.reservations.devices`(`driver: nvidia`)를 씁니다. 호스트 사전 준비는 아래 "GPU 가속" 절 참고.
|
||||
|
||||
`docker compose up` 한 번이면 자동으로:
|
||||
|
||||
@@ -1,33 +1,112 @@
|
||||
// True-mode browser action core. Drives the on-screen Chrome (CDP at CDP_PORT,
|
||||
// default 9222) so the action is visible on the Go-Live broadcast, and prints a
|
||||
// JSON result on stdout for the Python `browseAndSearch` tool to wrap.
|
||||
// Browser action core. Prefers the on-screen Chrome (CDP at CDP_PORT, default
|
||||
// 9222) so the action is visible on the Go-Live broadcast, and prints a JSON
|
||||
// result on stdout for the Python `browseAndSearch` tool to wrap.
|
||||
//
|
||||
// node browse-search.mjs "<query>" [search|youtube]
|
||||
//
|
||||
// - search : Google-search the query, return the top organic results.
|
||||
// - youtube : search YouTube and play the first result.
|
||||
//
|
||||
// Backend selection for `search`:
|
||||
// 1. The broadcast Chrome over CDP (visible on the Go-Live stream).
|
||||
// 2. Else, if CHROME_USER_DATA_DIR is set, a persistent Chrome using that
|
||||
// profile dir. Logging that dedicated profile into Google once lets Google
|
||||
// treat later searches as a returning signed-in user, which avoids the
|
||||
// bot-detection interstitial that blocks a fresh anonymous session.
|
||||
// 3. Else a fresh ephemeral headless Chrome (works only where Google does not
|
||||
// challenge the session, e.g. a non-flagged residential IP).
|
||||
// `youtube` only makes sense on the visible broadcast Chrome, so it never uses
|
||||
// the headless/persistent fallback.
|
||||
import { chromium } from 'playwright';
|
||||
|
||||
const CDP = process.env.CDP_PORT || '9222';
|
||||
// Use 127.0.0.1, not "localhost": in containers localhost can resolve to IPv6
|
||||
// (::1) first while Chrome's CDP listens on IPv4, giving ECONNREFUSED ::1.
|
||||
const CDP_HOST = process.env.CDP_HOST || '127.0.0.1';
|
||||
const USER_DATA_DIR = process.env.CHROME_USER_DATA_DIR || '';
|
||||
const UA =
|
||||
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
|
||||
'(KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36';
|
||||
const query = process.argv[2] || '';
|
||||
const mode = (process.argv[3] || 'search').toLowerCase();
|
||||
const out = (o) => { process.stdout.write(JSON.stringify(o)); };
|
||||
|
||||
if (!query) { out({ ok: false, error: 'no query' }); process.exit(1); }
|
||||
|
||||
let b;
|
||||
let connected; // CDP Browser (the broadcast Chrome — never kill it)
|
||||
let launchedBrowser; // ephemeral headless Browser we launched
|
||||
let persistent; // persistent BrowserContext we launched
|
||||
let launched = false;
|
||||
let page;
|
||||
|
||||
// Try system Chrome (channel:'chrome') first so no extra Playwright browser
|
||||
// download is needed; fall back to Playwright's bundled chromium.
|
||||
async function tryLaunch(launchFn) {
|
||||
let err;
|
||||
for (const opts of [{ headless: true, channel: 'chrome' }, { headless: true }]) {
|
||||
try {
|
||||
b = await chromium.connectOverCDP(`http://${CDP_HOST}:${CDP}`);
|
||||
const ctx = b.contexts()[0];
|
||||
const page = ctx.pages()[0] || (await ctx.newPage());
|
||||
return await launchFn(opts);
|
||||
} catch (e) {
|
||||
err = e;
|
||||
}
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
|
||||
async function acquirePage() {
|
||||
// 1. Broadcast Chrome over CDP.
|
||||
try {
|
||||
connected = await chromium.connectOverCDP(`http://${CDP_HOST}:${CDP}`);
|
||||
const ctx = connected.contexts()[0];
|
||||
page = ctx.pages()[0] || (await ctx.newPage());
|
||||
return;
|
||||
} catch (e) {
|
||||
if (mode === 'youtube') throw e; // youtube needs the visible broadcast Chrome
|
||||
}
|
||||
|
||||
// 2. Persistent profile (signed-in) when configured.
|
||||
if (USER_DATA_DIR) {
|
||||
persistent = await tryLaunch((opts) =>
|
||||
chromium.launchPersistentContext(USER_DATA_DIR, { ...opts, locale: 'ko-KR', userAgent: UA }),
|
||||
);
|
||||
launched = true;
|
||||
page = persistent.pages()[0] || (await persistent.newPage());
|
||||
return;
|
||||
}
|
||||
|
||||
// 3. Ephemeral headless.
|
||||
launchedBrowser = await tryLaunch((opts) => chromium.launch(opts));
|
||||
launched = true;
|
||||
const ctx = await launchedBrowser.newContext({ locale: 'ko-KR', userAgent: UA });
|
||||
page = await ctx.newPage();
|
||||
}
|
||||
|
||||
async function closeAll() {
|
||||
try { await persistent?.close(); } catch { /* ignore */ }
|
||||
try { await launchedBrowser?.close(); } catch { /* ignore */ }
|
||||
try { await connected?.close(); } catch { /* ignore */ }
|
||||
}
|
||||
|
||||
// Human-like search: land on the site's home page, type the query into its
|
||||
// search box one key at a time, and press Enter — the way a person would,
|
||||
// rather than jumping straight to a results URL.
|
||||
async function typeSearch(homeUrl, boxSelector, query) {
|
||||
await page.goto(homeUrl, { waitUntil: 'domcontentloaded' });
|
||||
const box = page.locator(boxSelector).first();
|
||||
await box.waitFor({ timeout: 15000 });
|
||||
await box.click();
|
||||
await box.pressSequentially(query, { delay: 45 });
|
||||
await box.press('Enter');
|
||||
}
|
||||
|
||||
try {
|
||||
await acquirePage();
|
||||
page.setDefaultTimeout(20000);
|
||||
await page.bringToFront().catch(() => {});
|
||||
|
||||
if (mode === 'youtube') {
|
||||
await page.goto(`https://www.youtube.com/results?search_query=${encodeURIComponent(query)}`, { waitUntil: 'domcontentloaded' });
|
||||
// Type into YouTube's search box like a person, then play the first result.
|
||||
await typeSearch('https://www.youtube.com/?hl=ko', 'input#search, input[name="search_query"]', query);
|
||||
await page.waitForSelector('ytd-video-renderer a#video-title, a#video-title', { timeout: 20000 });
|
||||
const first = page.locator('ytd-video-renderer a#video-title, a#video-title').first();
|
||||
const title = (await first.getAttribute('title').catch(() => '')) || (await first.innerText().catch(() => ''));
|
||||
@@ -36,8 +115,19 @@ try {
|
||||
await page.evaluate(() => { const v = document.querySelector('video'); if (v && v.paused) v.play(); });
|
||||
out({ ok: true, mode, title: (title || '').trim(), url: page.url() });
|
||||
} else {
|
||||
await page.goto(`https://www.google.com/search?q=${encodeURIComponent(query)}&hl=ko`, { waitUntil: 'domcontentloaded' });
|
||||
// Type into Google's search box like a person, then read the results.
|
||||
await typeSearch('https://www.google.com/?hl=ko', 'textarea[name="q"], input[name="q"]', query);
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
await page.waitForTimeout(1500);
|
||||
// Google serves its bot-detection interstitial (/sorry/index) to sessions it
|
||||
// suspects are automated. Detect it structurally (by URL, locale-independent)
|
||||
// and fail fast so the Python caller fail-opens to the DDG cascade instead of
|
||||
// treating an empty challenge page as "no results".
|
||||
if (page.url().includes('/sorry/')) {
|
||||
await closeAll();
|
||||
out({ ok: false, error: 'google-bot-challenge', headless: launched });
|
||||
process.exit(1);
|
||||
}
|
||||
const results = await page.evaluate(() => {
|
||||
const seen = new Set();
|
||||
const items = [];
|
||||
@@ -55,11 +145,11 @@ try {
|
||||
}
|
||||
return items;
|
||||
});
|
||||
out({ ok: true, mode, query, count: results.length, results });
|
||||
out({ ok: true, mode, query, count: results.length, results, headless: launched });
|
||||
}
|
||||
await b.close();
|
||||
await closeAll();
|
||||
} catch (e) {
|
||||
try { await b?.close(); } catch { /* ignore */ }
|
||||
await closeAll();
|
||||
out({ ok: false, error: String(e?.message || e) });
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -5,8 +5,9 @@
|
||||
#
|
||||
# docker compose -f docker-compose.yml -f docker-compose.gpu-windows.yml up -d
|
||||
#
|
||||
# Or set COMPOSE_FILE in .env (recommended):
|
||||
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml
|
||||
# Or set COMPOSE_FILE in .env (note the ";" separator on Windows — ":" collides
|
||||
# with the C: drive letter and breaks file resolution):
|
||||
# COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml
|
||||
services:
|
||||
ollama:
|
||||
deploy:
|
||||
|
||||
@@ -3,6 +3,12 @@
|
||||
One image, three roles (`JARVIS_ROLE`), selected in `.env`. GPU is added per OS
|
||||
via a compose override picked with `COMPOSE_FILE`.
|
||||
|
||||
> `COMPOSE_FILE`'s file separator is OS-specific: Linux/macOS use `:`, Windows
|
||||
> uses `;` (a colon collides with the `C:` drive letter). Using `:` on Windows
|
||||
> yields `... The system cannot find the file specified`. If in doubt, leave
|
||||
> `COMPOSE_FILE` unset and pass the files explicitly:
|
||||
> `docker compose -f docker-compose.yml -f docker-compose.gpu-windows.yml up -d`.
|
||||
|
||||
## A. All-in-one (single machine)
|
||||
|
||||
Everything (desktop + Chrome + bridge + bot + TTS) in one container.
|
||||
@@ -10,8 +16,8 @@ Everything (desktop + Chrome + bridge + bot + TTS) in one container.
|
||||
```
|
||||
# .env
|
||||
JARVIS_ROLE=full
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu
|
||||
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml # Windows 11
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu/macOS (":" )
|
||||
# COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml # Windows 11 (";" )
|
||||
DISCORD_SELFBOT_TOKEN=...
|
||||
DISCORD_GUILD_ID=...
|
||||
|
||||
@@ -45,8 +51,8 @@ Watch it on this machine’s VNC (`localhost:5901`) / noVNC (`localhost:6080`).
|
||||
# .env
|
||||
JARVIS_ROLE=bot
|
||||
BROWSER_CONTROL_URL=http://192.168.10.9:8777 # the browser host's LAN IP
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu
|
||||
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml # Windows 11
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu/macOS (":" )
|
||||
# COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml # Windows 11 (";" )
|
||||
DISCORD_SELFBOT_TOKEN=...
|
||||
DISCORD_GUILD_ID=...
|
||||
|
||||
@@ -68,6 +74,11 @@ human-style input (visible on its VNC).
|
||||
compose runs from PowerShell/cmd). Seed it once from a machine with a browser and
|
||||
the logged-in Gemini CLI (`npm i -g @google/gemini-cli`, then `gemini` ->
|
||||
"Sign in with Google"), copying the login state:
|
||||
(Note: as of 2026-06 Google blocks personal Google accounts on this CLI login
|
||||
with "This client is no longer supported for Gemini Code Assist for
|
||||
individuals". Workspace/org accounts may still work; personal accounts should
|
||||
use `GEMINI_AUTH=apikey` with a key from https://aistudio.google.com/app/apikey
|
||||
instead. Real-time search fail-opens to DDG/Brave/Wikipedia either way.)
|
||||
`cp -r ~/.gemini/. docker/gemini-oauth/`. The essential file is `oauth_creds.json`
|
||||
(it holds the refresh token; `GOOGLE_GENAI_USE_GCA=true` forces OAuth, so that is
|
||||
the file the startup readiness check looks for) - copying the whole dir simply also
|
||||
|
||||
@@ -22,7 +22,29 @@ path (evals, text entry) and falls back to the master flag:
|
||||
|
||||
- **on-screen Chrome**: `browser_search()` drives Chrome (Node CDP helper
|
||||
`bot/scripts/stream-test/browse-search.mjs`) to Google-search the query, so
|
||||
the action is visible on the Go-Live broadcast.
|
||||
the action is visible on the Go-Live broadcast. The helper searches the
|
||||
human way — it loads the site home page, types the query into the search box
|
||||
one key at a time, and presses Enter (both Google `search` and `youtube`),
|
||||
rather than jumping to a results URL. When no broadcast Chrome is
|
||||
reachable on CDP (e.g. a plain text turn with no active broadcast), the helper
|
||||
falls back, for `search` only, to launching its own Chrome so browser-based
|
||||
Google search still works with no API cost. Fallback order:
|
||||
- **CDP** (the broadcast Chrome) — preferred, visible on the stream.
|
||||
- **Persistent profile** when `CHROME_USER_DATA_DIR` is set — Chrome opened
|
||||
against that profile dir (system `channel: 'chrome'`, else bundled chromium).
|
||||
Logging that dedicated profile into Google once lets Google treat later
|
||||
searches as a returning signed-in user, which is what avoids the
|
||||
bot-detection interstitial. This is the reliable way to get browser Google
|
||||
search in plain text turns.
|
||||
- **Ephemeral headless** otherwise — a fresh anonymous session; works only
|
||||
where Google does not challenge it (e.g. a non-flagged residential IP).
|
||||
|
||||
The `youtube` action never uses the fallback (it only makes sense on the
|
||||
visible broadcast Chrome). Caveat: an anonymous (not-signed-in) session can be
|
||||
served Google's bot-detection interstitial (`/sorry/index`); the helper
|
||||
detects this structurally by URL and fails fast, so the caller fail-opens to
|
||||
the DDG / Brave / Wikipedia cascade rather than treating the challenge page as
|
||||
"no results".
|
||||
- **Gemini**: answers, with the sub-mode chosen by `cfg.gemini_auth`
|
||||
(env `GEMINI_AUTH`, default `oauth`):
|
||||
- `oauth` (default): `gemini_cli_search()` shells out to the Gemini CLI
|
||||
|
||||
Reference in New Issue
Block a user