27 Commits

Author SHA1 Message Date
javis-bot
ccddbd6448 test: settings output_language survives save→apply→recreate
Some checks failed
Release / semantic-release (push) Successful in 42s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / build-linux (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
tests / Unit tests (Linux, Python 3.11) (push) Has been cancelled
Integration test driving the real bridge _save() and engine
_resolve_output_language(): a language chosen in the settings UI is written to
both the persistent volume and the runtime config, applies immediately (config
wins over the OUTPUT_LANGUAGE env), and survives a simulated container recreate
(entrypoint re-renders the config then merges the persistent override). Also
asserts the persona and reply directive both follow the persisted language.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 19:56:13 +09:00
javis-bot
7870a76314 fix: persona uses settings output_language, matching the reply directive
The persona prompt was built from the raw OUTPUT_LANGUAGE env while the
reply-language directive read the settings-web UI value (config JSON), so
changing the language in the settings page was honoured by the directive
but ignored by the persona, leaving them contradicting each other.

Add _resolve_output_language() as the single source of truth (config wins
over env) and feed the same resolved value to both build_system_prompt()
and reply_language_directive(). Update docs/llm_contexts.md to match.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 19:47:26 +09:00
javis-bot
b3088dd75f fix: settings output_language overrides the compose env default
The settings-UI output_language was ignored because the OUTPUT_LANGUAGE env took
precedence. Read the config value first, fall back to env, so changing the
language in /settings actually applies.
2026-06-15 16:57:01 +09:00
javis-bot
8868381f13 fix: minimal Chrome flags (drop --test-type/AutomationControlled) + policy infobar suppress
Per the flag hypothesis: remove the automation-signaling flags that can trigger
Google's /sorry/ challenge. Suppress the --no-sandbox warning bar with a Chrome
managed policy (CommandLineFlagSecurityWarningsEnabled=false) instead of
--test-type, so the infobar stays hidden without the automation signal.
2026-06-15 16:51:11 +09:00
javis-bot
247edda3eb fix: google anti-bot flag + persistent/safe settings apply + TTS engine wiring
- Chrome: --disable-blink-features=AutomationControlled (+ ko-KR) so Google
  shows results, not the /sorry/ automation block.
- Settings persist to /data/jarvis-settings.json (survives recreate; entrypoint
  re-merges it) AND the runtime config; apply restarts via a DETACHED process so
  the HTTP response isn't dropped when the bridge restarts.
- Bridge reads tts_engine from the settings config so the TTS-engine choice
  actually applies.
2026-06-15 13:13:11 +09:00
javis-bot
84e435f916 feat: settings web UI (models / STT / TTS speed / language / LLM instructions)
Adds /settings (served by the bridge) to change the LLM model (from installed
Ollama models), Whisper model, TTS engine + MeloTTS speed, output language,
agentic max-turns, thinking mode, and free-form LLM instructions — live, with a
'apply' that restarts the bridge + TTS worker. Settings persist to the runtime
config JSON; engine reads output_language + llm_instructions and the TTS worker
reads melo_speed from it. Bridge port publishable for access.
2026-06-15 13:05:46 +09:00
javis-bot
3bdc7d078a feat: cross-platform compose (Ubuntu CDI + Windows Docker Desktop GPU)
Base compose is GPU-agnostic; GPU is added by a per-OS override selected via
COMPOSE_FILE in .env (docker-compose.gpu-linux.yml for Ubuntu/CDI,
docker-compose.gpu-windows.yml for Windows 11 Docker Desktop). Adds .env.example
split-deployment section + docs/DEPLOY.md covering all-in-one and browser+bot
layouts on both OSes.
2026-06-15 13:00:04 +09:00
javis-bot
8dd6386af8 fix: search like a person — open homepage, type in the site's search box
controlBrowser search opened the results URL directly (search.naver.com?query).
Now it navigates to the homepage (www.naver.com / google.com), clicks the
on-page search box, types the query char-by-char and presses Enter — real
human-style search, visible on screen.
2026-06-15 12:42:58 +09:00
javis-bot
aebf183950 feat: browser-control server on host (real input) + remote-bot routing + ignore env backups
- control-server.mjs runs chrome-control.mjs LOCALLY on the browser host, so a
  remote bot's controlBrowser (BROWSER_CONTROL_URL) drives real xdotool input
  on THIS screen instead of the bot machine. Published on the LAN.
- controlBrowser tool posts to BROWSER_CONTROL_URL when set, else runs locally.
- Drop hard depends_on ollama so a browser-host doesn't start Ollama.
- gitignore .env.bak*/*.bak (a backup with tokens had been left untracked).
2026-06-15 10:41:57 +09:00
javis-bot
1935c1a6bc feat: split-deployment roles (browser-host on LAN + remote bot)
Add JARVIS_ROLE (full|browser|bot) via a run-if-role.sh supervisord guard so
one image serves three layouts. Make Chrome CDP bind configurable (CDP_BIND)
and publishable on the LAN (CDP_PUBLISH_BIND) so a bot on another PC can drive
this host's on-screen Chrome over the internal network (no auth, as requested).
2026-06-15 10:23:55 +09:00
javis-bot
bdb012fc7c fix: weather passes named city from utterance; clean navigate reply
GeoIP auto-detect is unavailable in the container, so '부산 날씨' failed
(no location). Extract the city from the utterance and pass it to getWeather.
Also report navigate by site name instead of the mid-load about:blank url.
2026-06-15 01:02:35 +09:00
javis-bot
49061d30f0 fix: deterministic browser navigate for 'go back to <site>' (구글로 돌아가)
The 7B narrated '구글 메인으로 돌아갑니다' without acting, so the screen stayed on
Naver. Split site intent into SEARCH vs NAV: nav words (돌아가/이동/열어/메인/go
back) now drive controlBrowser.navigate to the site homepage directly, search
words run controlBrowser.search — both deterministically, no LLM.
2026-06-15 00:58:53 +09:00
javis-bot
c522e1b285 fix: deterministic weather → one clean Korean sentence (no 'Celsius')
getWeather now returns only the Korean sentence (지금 <곳> 날씨는 <상태>, 기온 N도
(체감 M도)입니다) with no English/°C source. A deterministic weather path in the
engine returns it verbatim, bypassing the 7B which was rephrasing into multiple
sentences and leaking 'Celsius'.
2026-06-14 22:46:03 +09:00
javis-bot
54c3ce7d1b feat: show speaker nickname instead of raw user ID in voice logs
Resolve the Discord user ID to a server nickname / global name (cached) and
display that in the transcript channel + console logs.
2026-06-14 22:39:06 +09:00
javis-bot
d970bf276e fix: harden Korean-only output lock (front+end, explicit script ban)
qwen2.5:7b leaked Chinese/Cyrillic mid-reply despite the OUTPUT_LANGUAGE
lock, which was buried mid-prompt. Repeat the lock at the END of the system
prompt (recency) and ban specific foreign scripts explicitly.
2026-06-14 22:25:33 +09:00
javis-bot
3d620dc4c7 fix: concise Korean weather reply (current conditions, one sentence)
getWeather returned a verbose multi-section English forecast that the 3B
re-synthesised into long, CJK/°F-leaking answers. Hand it a ready-to-speak
Korean one-liner (지금 <곳> 날씨는 <상태>, 기온 N도(체감 M도)입니다) and drop the
hourly/7-day firehose from the default voice reply.
2026-06-14 21:49:53 +09:00
javis-bot
09afc21283 feat: split recording vs STT-processing time in turn logs
Log the captured speech-clip duration (녹음/음성) separately from the Whisper
transcription time (STT처리) so it's clear whether a slow turn is the
listening/recording or the transcription, per the user's request.
2026-06-14 21:45:50 +09:00
javis-bot
11a72cb296 fix: deterministic on-screen site search + lock replies to Korean
- Site-specified search ("네이버에서 X 검색해줘") now runs controlBrowser.search
  directly in the engine when broadcasting, instead of relying on the 3B model
  to emit the tool call (it kept narrating "검색하겠습니다" without acting).
- Set OUTPUT_LANGUAGE=ko so replies are Korean-only — stops the small model
  leaking CJK/Hanja and English fragments (每, 朗, "feels like") into weather
  and other answers, and keeps them concise.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 21:07:26 +09:00
javis-bot
37759f2b2c perf: conversational fast-path (skip enrichment) + shorter silence wait
Greetings/small-talk routed no data tool yet still ran the episodic memory
enrichment (LLM keyword extract + diary/graph search, ~1s) every turn. Skip it
when the router picked no external-data tool — the always-injected warm profile
still personalises the reply. Also drop the voice silence-detection wait
800ms -> 600ms for snappier turn-taking. Warm "안녕" now lands well under the
3-4s target.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 03:21:52 +09:00
javis-bot
4bc7f836ae fix: native tool-calling for qwen2.5 (tools actually fire) + kill Chrome infobars
Root cause of "weather/search do nothing": the engine forced TEXT tool-calling
for all <=7B models, but qwen2.5:3b emits clean NATIVE tool calls and fails at
the text format — so it just narrated ("부산 날씨는 맑습니다") and never called
getWeather/webSearch/controlBrowser. Use native tool-calling for tool-capable
small families (qwen2.5/qwen3/llama3.x/mistral); native still auto-falls back
to text on HTTP 400, so non-tool models (gemma) are unaffected.

Also launch Chrome with --test-type (removes the "--no-sandbox unsupported
flag" infobar) and disable the Translate feature/popup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 03:08:49 +09:00
javis-bot
642fa42561 fix: drop webSearch when a site is named in screen-share, forcing controlBrowser
The 3B model kept choosing webSearch over controlBrowser even when offered, so
'네이버에서 X 검색' still used the invisible web path. When broadcasting and the
user explicitly names a site, remove webSearch from the allow-list so the
on-screen browser is the only search route.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:57:51 +09:00
javis-bot
f3d34bab4d fix: always offer controlBrowser in screen-share so on-screen search works
The small router reflexively routed every "search/open" intent to webSearch
and never surfaced controlBrowser, so "네이버에서 X 검색해줘" did nothing on the
broadcast. Union controlBrowser (+browseAndPlay) into the allow-list every
turn in screen-share mode (like setBroadcast), and steer the model in the
system prompt to prefer the on-screen browser over webSearch when available.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:55:33 +09:00
javis-bot
c21c5b225f feat: controlBrowser 'search' action + one-sentence voice replies
- Add a one-shot `search` action (site=naver/google/daum/youtube/bing) that
  navigates the on-screen browser straight to the results page, so a small
  model can satisfy "search X on Naver" in a single tool call instead of a
  fragile navigate->type->enter chain.
- Sharpen the tool description to steer the router to controlBrowser (not
  webSearch) for anything that should happen IN the visible browser.
- System prompt: answer in one short sentence (voice assistant) — also cuts
  TTS time.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:52:47 +09:00
javis-bot
109dbc7e16 feat: robust controlBrowser — real xdotool input, tabs/back/forward/popups
- Install xdotool + wmctrl (late Docker layer, preserves the melo cache) so
  the on-screen Chrome gets real X input (visible cursor, char-by-char typing)
  instead of synthetic events; falls back to the Playwright API if absent.
- Fix active-tab detection (probe document.visibilityState instead of assuming
  tab 0) so sequential ops target the right tab.
- Add back / forward / refresh; new/switch/close tabs via real keyboard
  (Ctrl+T / Ctrl+<n> / Ctrl+W) when xdotool is present.
- Auto-dismiss native JS dialogs; closePopups clears blank/popup tabs.
- Report broadcast (Go-Live) state in status from the turn context.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:33:59 +09:00
javis-bot
b18217fcdd fix: let melo-worker honour MELO_DEVICE from env (was hardcoded cpu)
supervisord hardcoded MELO_DEVICE=cpu, overriding the compose MELO_DEVICE=cuda
so MeloTTS stayed on CPU even after the GPU torch swap. Interpolate the
container env instead.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:26:24 +09:00
javis-bot
3d1e56f60f feat: controlBrowser tool — human-operable on-screen Chrome
Adds a general browser-control tool (navigate to any site, list/open/close/
switch tabs, close popups, click, type, scroll, screenshot) for the Go-Live
Chrome, on top of the existing CDP + xdotool human-input layer (visible
cursor, char-by-char typing). Closes the gap where "open Naver" had no tool
and the model confabulated success. Also adds a system-prompt rule against
claiming actions no tool actually performed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:22:36 +09:00
javis-bot
927d59f805 perf: run MeloTTS on the GPU (cu128 torch) + warm CUDA at startup
CPU MeloTTS serialised under concurrent load (whisper STT + bot) and blew
voice-reply TTS to 7-8s. Install the Blackwell-verified cu128 torch in the
melo venv, select the GPU via MELO_DEVICE=cuda, and do a throwaway synth at
worker startup so the one-off CUDA kernel-init (~5s) doesn't land on the
user's first reply. Measured: ~0.3s/sentence on GPU vs ~1.2-2.6s on CPU.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:22:36 +09:00
27 changed files with 1582 additions and 107 deletions

View File

@@ -152,3 +152,45 @@ SCREENSHOT_INTERVAL_SEC=5
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Silence (ms) that marks the end of an utterance before sending to the brain. # Silence (ms) that marks the end of an utterance before sending to the brain.
VOICE_SILENCE_MS=800 VOICE_SILENCE_MS=800
# ===========================================================================
# Split deployment & cross-platform (Ubuntu + Windows 11)
# ===========================================================================
# JARVIS_ROLE selects what this machine runs (see docker/run-if-role.sh):
# full (default) everything in one container
# browser ONLY the desktop + Chrome + control-server (driven over the LAN)
# bot ONLY the bot + bridge + TTS (drives a REMOTE browser)
JARVIS_ROLE=full
# --- GPU per OS: pick the matching compose override via COMPOSE_FILE ---
# Ubuntu (nvidia-container-toolkit / CDI):
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
# Windows 11 (Docker Desktop + WSL2 + NVIDIA):
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml
# Browser-only host (no GPU needed): leave COMPOSE_FILE unset (base only).
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
# --- Browser HOST (JARVIS_ROLE=browser) — e.g. this LAN machine ---
# Expose Chrome control to the internal network (no auth, internal only):
# CDP_BIND=0.0.0.0
# BROWSER_CONTROL_BIND=0.0.0.0
# CDP_PUBLISH_BIND=0.0.0.0
# Defaults are loopback-only.
# --- BOT host (JARVIS_ROLE=bot) — e.g. your PC driving the remote browser ---
# Point the controlBrowser tool at the browser host's control-server:
# BROWSER_CONTROL_URL=http://192.168.10.9:8777
# (Leave BROWSER_CONTROL_URL empty on full/browser layouts.)
# --- Models (tune per machine) ---
# OLLAMA_CHAT_MODEL=qwen2.5:7b # quality (needs ~5GB VRAM + whisper small)
# OLLAMA_CHAT_MODEL=qwen2.5:3b # speed (fits easily, faster on 8GB GPUs)
# WHISPER_MODEL=small # small frees VRAM for a bigger LLM; medium=more accurate
# MELO_DEVICE=cuda # cpu if no GPU on the bot host
# --- Settings web UI (http://localhost:8765/settings on the bot host) ---
# To reach it, expose the bridge to the host loopback:
# BRIDGE_HOST=0.0.0.0
# SETTINGS_PUBLISH_BIND=127.0.0.1 # 0.0.0.0 to allow LAN access (no auth)
# Change models / STT / TTS speed / language / LLM instructions live; "적용"
# restarts the bridge + TTS worker so changes take effect.

3
.gitignore vendored
View File

@@ -25,3 +25,6 @@ qt.conf
# Auto-generated version file (created at build time) # Auto-generated version file (created at build time)
src/jarvis/_version.py src/jarvis/_version.py
# never commit env backups (contain tokens)
.env.bak*
*.bak

View File

@@ -65,6 +65,14 @@ RUN ls -d /opt/venv/lib/python*/site-packages/nvidia/cublas/lib \
COPY docker/setup-melo.sh /app/docker/setup-melo.sh COPY docker/setup-melo.sh /app/docker/setup-melo.sh
RUN bash /app/docker/setup-melo.sh RUN bash /app/docker/setup-melo.sh
# --- Human input + window management for the on-screen Chrome control tool.
# Placed AFTER the heavy melo layer so it doesn't bust that cache. xdotool
# injects real X pointer/keyboard events (visible cursor, char-by-char
# typing) into the broadcast; wmctrl lists/moves windows. ---
RUN apt-get update && apt-get install -y --no-install-recommends \
xdotool wmctrl \
&& rm -rf /var/lib/apt/lists/*
# --- Discord bot deps (cache layer on lockfile) --- # --- Discord bot deps (cache layer on lockfile) ---
COPY bot/package.json bot/bun.lock /app/bot/ COPY bot/package.json bot/bun.lock /app/bot/
RUN cd /app/bot && bun install --frozen-lockfile || bun install RUN cd /app/bot && bun install --frozen-lockfile || bun install

View File

@@ -0,0 +1,269 @@
// Human-operable Chrome control for the on-screen (Go-Live) browser.
//
// node chrome-control.mjs '<json-command>'
//
// Connects to the on-screen Chrome over CDP (so every action is visible on the
// broadcast) and performs ONE command, printing a JSON result on stdout for the
// Python `controlBrowser` tool to wrap.
//
// Input style: when xdotool is available the pointer/keyboard ACTIONS
// (navigate via the omnibox, click, type, scroll, tab keys) are driven as REAL
// X input — the cursor visibly moves and text is typed one character at a time,
// exactly as a person would. If xdotool is missing it falls back to the
// Playwright/CDP API so the action still happens (just without a visible
// cursor). READ actions always use the CDP/DOM API.
//
// Commands (json): { "action": "<name>", ...params }
// status | listTabs
// navigate {url} | back | forward | refresh
// newTab {url?} | closeTab {index?} | activateTab {index} | closePopups
// click {selector} | type {text, selector?} | scroll {dir, notches?}
// pressKey {key} | screenshot {path}
import { chromium } from 'playwright';
import { execFileSync } from 'node:child_process';
import * as human from './human.mjs';
const CDP = process.env.CDP_PORT || '9222';
const CDP_HOST = process.env.CDP_HOST || '127.0.0.1';
const out = (o) => process.stdout.write(JSON.stringify(o));
const HAS_XDOTOOL = (() => {
try { execFileSync('which', ['xdotool'], { stdio: 'ignore' }); return true; }
catch { return false; }
})();
let cmd;
try { cmd = JSON.parse(process.argv[2] || '{}'); }
catch (e) { out({ ok: false, error: `bad command json: ${e?.message || e}` }); process.exit(1); }
const action = String(cmd.action || '').trim();
if (!action) { out({ ok: false, error: 'no action' }); process.exit(1); }
const norm = (u) => (/^https?:\/\//i.test(u) ? u : `https://${u}`);
// The genuinely-active tab is the one whose document is visible. Playwright has
// no "active page" accessor over CDP, so probe visibilityState (fixes treating
// tab 0 as active and breaking sequential ops on a specific tab).
async function pickActive(pages) {
for (const p of pages) {
try { if (await p.evaluate(() => document.visibilityState === 'visible')) return p; }
catch { /* page may be closing */ }
}
return pages.find((p) => p.url() && p.url() !== 'about:blank') || pages[0];
}
async function tabInfo(pages, active) {
const list = [];
for (let i = 0; i < pages.length; i++) {
const p = pages[i];
list.push({ index: i, url: p.url(), title: await p.title().catch(() => ''), active: p === active });
}
return list;
}
let b;
try {
b = await chromium.connectOverCDP(`http://${CDP_HOST}:${CDP}`);
const ctx = b.contexts()[0];
if (!ctx) throw new Error('no browser context (is Chrome running?)');
let pages = ctx.pages();
if (!pages.length) pages = [await ctx.newPage()];
let page = await pickActive(pages);
page.setDefaultTimeout(20000);
// Auto-dismiss native JS dialogs (alert/confirm/beforeunload) so a popup can
// never wedge the page.
page.on('dialog', (d) => d.dismiss().catch(() => {}));
const front = async (p) => { await p.bringToFront().catch(() => {}); };
const reload = async () => { await page.reload({ waitUntil: 'domcontentloaded' }).catch(() => {}); };
switch (action) {
case 'status':
case 'listTabs':
await front(page);
out({ ok: true, browserOpen: true, xdotool: HAS_XDOTOOL, tabCount: pages.length, tabs: await tabInfo(pages, page) });
break;
case 'navigate': {
const url = String(cmd.url || '').trim();
if (!url) throw new Error('navigate: no url');
await front(page);
if (HAS_XDOTOOL && cmd.human !== false) {
try { await human.navigateOmnibox(norm(url)); await page.waitForLoadState('domcontentloaded').catch(() => {}); }
catch { await page.goto(norm(url), { waitUntil: 'domcontentloaded' }); }
} else {
await page.goto(norm(url), { waitUntil: 'domcontentloaded' });
}
out({ ok: true, url: page.url(), title: await page.title().catch(() => ''), input: HAS_XDOTOOL ? 'human' : 'api' });
break;
}
case 'search': {
// Search like a PERSON: open the site's main page, click its search box,
// type the query char-by-char, press Enter — NOT a direct results-URL.
const q = String(cmd.query || '').trim();
if (!q) throw new Error('search: no query');
const siteKey = String(cmd.site || 'google').toLowerCase();
const SITES = {
naver: { home: 'https://www.naver.com', box: '#query, input[name="query"]' },
google: { home: 'https://www.google.com', box: 'textarea[name="q"], input[name="q"]' },
daum: { home: 'https://www.daum.net', box: '#q, input[name="q"]' },
youtube: { home: 'https://www.youtube.com', box: 'input#search, input[name="search_query"]' },
bing: { home: 'https://www.bing.com', box: '#sb_form_q, input[name="q"]' },
};
const s = SITES[siteKey] || SITES.google;
await front(page);
// 1) Go to the homepage.
if (HAS_XDOTOOL && cmd.human !== false) {
try { await human.navigateOmnibox(s.home); await page.waitForLoadState('domcontentloaded').catch(() => {}); }
catch { await page.goto(s.home, { waitUntil: 'domcontentloaded' }); }
} else {
await page.goto(s.home, { waitUntil: 'domcontentloaded' });
}
// 2) Click the on-page search box, type the query, submit.
const box = page.locator(s.box).first();
await box.waitFor({ state: 'visible', timeout: 15000 }).catch(() => {});
if (HAS_XDOTOOL && cmd.human !== false) {
try {
await human.humanClick(page, box);
await human.humanType(q);
await human.pressKey('Return');
} catch {
await box.click().catch(() => {});
await box.fill(q).catch(() => {});
await page.keyboard.press('Enter').catch(() => {});
}
} else {
await box.click().catch(() => {});
await box.fill(q);
await page.keyboard.press('Enter');
}
await page.waitForLoadState('domcontentloaded').catch(() => {});
out({ ok: true, site: SITES[siteKey] ? siteKey : 'google', query: q, url: page.url(), title: await page.title().catch(() => '') });
break;
}
case 'back': await front(page); await page.goBack({ waitUntil: 'domcontentloaded' }).catch(() => {}); out({ ok: true, url: page.url() }); break;
case 'forward': await front(page); await page.goForward({ waitUntil: 'domcontentloaded' }).catch(() => {}); out({ ok: true, url: page.url() }); break;
case 'refresh': await front(page); await reload(); out({ ok: true, url: page.url() }); break;
case 'newTab': {
let np;
if (HAS_XDOTOOL && cmd.human !== false) {
await front(page);
try { await human.pressKey('ctrl+t'); } catch { /* fall through */ }
await page.waitForTimeout(500);
const after = ctx.pages();
np = after[after.length - 1];
}
if (!np) np = await ctx.newPage(); // API fallback / no xdotool
await front(np);
if (cmd.url) {
if (HAS_XDOTOOL && cmd.human !== false) { try { await human.navigateOmnibox(norm(cmd.url)); } catch { await np.goto(norm(cmd.url)).catch(() => {}); } }
else await np.goto(norm(cmd.url), { waitUntil: 'domcontentloaded' }).catch(() => {});
}
out({ ok: true, index: ctx.pages().indexOf(np), url: np.url(), input: HAS_XDOTOOL ? 'human' : 'api' });
break;
}
case 'activateTab': {
const idx = Number(cmd.index);
if (!Number.isInteger(idx) || idx < 0 || idx >= pages.length) throw new Error('activateTab: bad index');
// Real keyboard: Ctrl+<1..8> selects that tab, Ctrl+9 the last.
if (HAS_XDOTOOL && cmd.human !== false && idx < 8) {
await front(pages[0]);
try { await human.pressKey(`ctrl+${idx + 1}`); } catch { await pages[idx].bringToFront(); }
} else {
await pages[idx].bringToFront();
}
out({ ok: true, active: idx, url: pages[idx].url() });
break;
}
case 'closeTab': {
const idx = Number.isInteger(Number(cmd.index)) ? Number(cmd.index) : pages.indexOf(page);
if (idx < 0 || idx >= pages.length) throw new Error('closeTab: bad index');
if (HAS_XDOTOOL && cmd.human !== false && idx < 8) {
await front(pages[0]);
try { await human.pressKey(`ctrl+${idx + 1}`); await human.pressKey('ctrl+w'); }
catch { await pages[idx].close(); }
} else {
await pages[idx].close();
}
out({ ok: true, closed: idx, remaining: ctx.pages().length });
break;
}
case 'closePopups': {
// Close popup / blank / extra tabs, keeping the active content tab.
let closed = 0;
for (const p of pages) {
if (p === page) continue;
const u = p.url();
if (cmd.all || !u || u === 'about:blank') { await p.close().catch(() => {}); closed++; }
}
out({ ok: true, closed, remaining: ctx.pages().length });
break;
}
case 'click': {
const selector = String(cmd.selector || '').trim();
if (!selector) throw new Error('click: no selector');
await front(page);
const locator = page.locator(selector).first();
if (HAS_XDOTOOL && cmd.human !== false) { try { await human.humanClick(page, locator); } catch { await locator.click(); } }
else await locator.click();
out({ ok: true });
break;
}
case 'type': {
const text = String(cmd.text ?? '');
await front(page);
if (cmd.selector) {
const locator = page.locator(String(cmd.selector)).first();
if (HAS_XDOTOOL && cmd.human !== false) { try { await human.humanClick(page, locator); } catch { await locator.click().catch(() => {}); } }
else { await locator.fill(text); out({ ok: true, input: 'api' }); break; }
}
if (HAS_XDOTOOL && cmd.human !== false) { try { await human.humanType(text); } catch { await page.keyboard.type(text, { delay: 80 }); } }
else await page.keyboard.type(text, { delay: 80 });
out({ ok: true, input: HAS_XDOTOOL ? 'human' : 'api' });
break;
}
case 'scroll': {
const dir = String(cmd.dir || 'down').toLowerCase() === 'up' ? -1 : 1;
if (HAS_XDOTOOL && cmd.human !== false) { try { await human.humanScroll(page, dir, Number(cmd.notches) || 5); } catch { await page.mouse.wheel(0, dir * 600); } }
else await page.mouse.wheel(0, dir * (Number(cmd.notches) || 5) * 120);
out({ ok: true });
break;
}
case 'pressKey': {
const key = String(cmd.key || '').trim();
if (!key) throw new Error('pressKey: no key');
if (HAS_XDOTOOL) { try { await human.pressKey(key); } catch { await page.keyboard.press(key); } }
else await page.keyboard.press(key);
out({ ok: true });
break;
}
case 'screenshot': {
const path = String(cmd.path || '').trim();
if (!path) throw new Error('screenshot: no path');
await front(page);
await page.screenshot({ path });
out({ ok: true, path });
break;
}
default:
out({ ok: false, error: `unknown action: ${action}` });
await b.close();
process.exit(1);
}
await b.close();
} catch (e) {
try { await b?.close(); } catch { /* ignore */ }
out({ ok: false, error: String(e?.message || e) });
process.exit(1);
}

View File

@@ -0,0 +1,48 @@
// Browser-control HTTP endpoint for the BROWSER HOST.
//
// The on-screen Chrome, the X display (:1), xdotool (real cursor/keyboard) and
// the broadcast capture all live on THIS machine. A remote `bot` on another PC
// therefore cannot drive them directly — it must send a command here, where
// chrome-control.mjs runs LOCALLY (real input lands on this host's screen,
// visible on its VNC / Go-Live).
//
// POST /control body: {"action":"navigate","url":"naver.com", ...}
// GET /health
//
// Internal-network use only (no auth, per deployment decision). Bind/port:
// BROWSER_CONTROL_BIND (default 0.0.0.0), BROWSER_CONTROL_PORT (default 8777)
import http from 'node:http';
import { execFile } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
const PORT = parseInt(process.env.BROWSER_CONTROL_PORT || '8777', 10);
const BIND = process.env.BROWSER_CONTROL_BIND || '0.0.0.0';
const SCRIPT = join(dirname(fileURLToPath(import.meta.url)), 'chrome-control.mjs');
const server = http.createServer((req, res) => {
if (req.method === 'GET' && req.url === '/health') {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ ok: true, host: 'browser' }));
return;
}
if (req.method !== 'POST') {
res.writeHead(405); res.end('POST /control');
return;
}
let body = '';
req.on('data', (c) => { body += c; if (body.length > 1e6) req.destroy(); });
req.on('end', () => {
// Run the action LOCALLY: chrome-control.mjs uses CDP + xdotool on this
// host, so the cursor really moves and text is typed on this screen.
execFile('node', [SCRIPT, body || '{}'], { timeout: 95_000, env: process.env }, (err, stdout, stderr) => {
res.writeHead(200, { 'Content-Type': 'application/json' });
const out = (stdout || '').trim();
res.end(out || JSON.stringify({ ok: false, error: String((stderr || '').trim() || err?.message || 'no output') }));
});
});
});
server.listen(PORT, BIND, () => {
console.log(`[control-server] listening on ${BIND}:${PORT} (browser host)`);
});

View File

@@ -38,6 +38,9 @@ export interface TurnInfo {
/** Discord user ID of the speaker, so the transcript shows whose audio /** Discord user ID of the speaker, so the transcript shows whose audio
* produced each turn (and which user a dropped/VAD turn belongs to). */ * produced each turn (and which user a dropped/VAD turn belongs to). */
user?: string; user?: string;
/** Resolved display name (server nickname / global name); shown instead of
* the raw user ID when available. */
userName?: string;
transcript: string; transcript: string;
reply: string; reply: string;
note?: string; note?: string;
@@ -72,7 +75,7 @@ function durSec(a?: number, b?: number): string | null {
* timing breakdown (listening / LLM / TTS) with start→end wall-clock times and * timing breakdown (listening / LLM / TTS) with start→end wall-clock times and
* durations, so it's obvious what took long. Pure + exported for testing. */ * durations, so it's obvious what took long. Pure + exported for testing. */
export function formatTurnMessage(info: TurnInfo): string { export function formatTurnMessage(info: TurnInfo): string {
const who = info.user ? `👤 ${info.user} ` : ""; const who = info.userName || info.user ? `👤 ${info.userName || info.user} ` : "";
const head = info.transcript const head = info.transcript
? `${who}🎤 들음 → 🗣️ "${info.transcript}"\n🤖 답변: ${(info.reply || "").trim() || "(무응답)"}` ? `${who}🎤 들음 → 🗣️ "${info.transcript}"\n🤖 답변: ${(info.reply || "").trim() || "(무응답)"}`
: `${who}🎤 들음 → ❌ ${info.note || "무시됨"}`; : `${who}🎤 들음 → ❌ ${info.note || "무시됨"}`;
@@ -124,7 +127,7 @@ async function joinAndListen(client: AnyClient, channelId: string): Promise<void
// joinVoiceChannel (it exposes id, guild.id and guild.voiceAdapterCreator). // joinVoiceChannel (it exposes id, guild.id and guild.voiceAdapterCreator).
const session = await joinChannel(channel as unknown as VoiceBasedChannel); const session = await joinChannel(channel as unknown as VoiceBasedChannel);
session.onTurn = (info) => { session.onTurn = (info) => {
console.log(`👤 ${info.user || "?"} 🗣️ ${info.transcript || "(" + (info.note || "empty") + ")"}\n🤖 ${info.reply}`); console.log(`👤 ${info.userName || info.user || "?"} 🗣️ ${info.transcript || "(" + (info.note || "empty") + ")"}\n🤖 ${info.reply}`);
// Mirror every heard utterance (and the reply / drop reason) to a text // Mirror every heard utterance (and the reply / drop reason) to a text
// channel so you can see what the bot understood even when it doesn't answer. // channel so you can see what the bot understood even when it doesn't answer.
void postTranscript(client, info); void postTranscript(client, info);

View File

@@ -81,6 +81,9 @@ export class VoiceSession {
* diagnosable. `note` says why (e.g. "음성 아님(VAD 차단)", "너무 짧음", "ok"). */ * diagnosable. `note` says why (e.g. "음성 아님(VAD 차단)", "너무 짧음", "ok"). */
onTurn?: (info: { onTurn?: (info: {
user: string; user: string;
/** Resolved display name (server nickname / global name) for the speaker,
* so logs show a human name instead of the raw Discord user ID. */
userName?: string;
transcript: string; transcript: string;
reply: string; reply: string;
note?: string; note?: string;
@@ -164,6 +167,31 @@ export class VoiceSession {
}); });
} }
/** Resolve a speaker's Discord user ID to a human display name (server
* nickname, else global name / username), cached so we don't refetch every
* utterance. Falls back to the ID if lookup fails. */
private nameCache = new Map<string, string>();
private async displayName(userId: string): Promise<string> {
const cached = this.nameCache.get(userId);
if (cached) return cached;
let name = userId;
try {
const guild: any = this.client?.guilds?.cache?.get(this.guildId);
let member: any = guild?.members?.cache?.get(userId);
if (!member && guild?.members?.fetch) member = await guild.members.fetch(userId).catch(() => null);
if (member) {
name = member.displayName || member.nickname || member.user?.globalName || member.user?.username || userId;
} else {
const u: any = this.client?.users?.cache?.get(userId) || (await this.client?.users?.fetch?.(userId).catch(() => null));
name = u?.globalName || u?.username || userId;
}
} catch {
/* fall back to id */
}
this.nameCache.set(userId, name);
return name;
}
private async captureUtterance(userId: string): Promise<void> { private async captureUtterance(userId: string): Promise<void> {
// Don't start a new capture once we're tearing down (user left). // Don't start a new capture once we're tearing down (user left).
if (this.destroyed) return; if (this.destroyed) return;
@@ -199,6 +227,7 @@ export class VoiceSession {
if (mono.length < DISCORD_RATE * 0.3 * 2) { if (mono.length < DISCORD_RATE * 0.3 * 2) {
this.onTurn?.({ this.onTurn?.({
user: userId, user: userId,
userName: await this.displayName(userId),
transcript: "", transcript: "",
reply: "", reply: "",
note: "너무 짧음(<300ms)", note: "너무 짧음(<300ms)",
@@ -247,6 +276,7 @@ export class VoiceSession {
// explains why a turn did or didn't answer, with full stage timing. // explains why a turn did or didn't answer, with full stage timing.
this.onTurn?.({ this.onTurn?.({
user: userId, user: userId,
userName: await this.displayName(userId),
transcript: metaSeen?.transcript ?? "", transcript: metaSeen?.transcript ?? "",
reply: metaSeen?.reply ?? "", reply: metaSeen?.reply ?? "",
note: metaSeen?.note, note: metaSeen?.note,

View File

@@ -36,7 +36,26 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
HOST = os.environ.get("MELO_WORKER_HOST", "127.0.0.1") HOST = os.environ.get("MELO_WORKER_HOST", "127.0.0.1")
PORT = int(os.environ.get("MELO_WORKER_PORT", "8770")) PORT = int(os.environ.get("MELO_WORKER_PORT", "8770"))
LANGUAGE = os.environ.get("MELO_LANGUAGE", "KR") LANGUAGE = os.environ.get("MELO_LANGUAGE", "KR")
SPEED = float(os.environ.get("MELO_SPEED", "1.5"))
def _resolve_speed() -> float:
"""Speaking rate: the settings-UI value (runtime config JSON) wins, else the
MELO_SPEED env, else 1.5. Read at startup; the settings UI restarts this
worker on apply so a new value takes effect."""
try:
cp = os.environ.get("JARVIS_CONFIG_PATH", "/app/config/jarvis.json")
v = json.loads(open(cp, encoding="utf-8").read()).get("melo_speed")
if v is not None:
return float(v)
except Exception:
pass
try:
return float(os.environ.get("MELO_SPEED", "1.5"))
except ValueError:
return 1.5
SPEED = _resolve_speed()
DEVICE = os.environ.get("MELO_DEVICE", "cpu") DEVICE = os.environ.get("MELO_DEVICE", "cpu")
# Model + speaker id are loaded once, guarded by a lock because MeloTTS # Model + speaker id are loaded once, guarded by a lock because MeloTTS
@@ -66,6 +85,20 @@ def _ensure_model() -> None:
speaker_id = spk_map[LANGUAGE] if LANGUAGE in spk_map else spk_map[keys[0]] speaker_id = spk_map[LANGUAGE] if LANGUAGE in spk_map else spk_map[keys[0]]
_model = model _model = model
_speaker_id = speaker_id _speaker_id = speaker_id
# Warm the GPU once at load: the first CUDA synth pays a one-off
# kernel-init cost (~5s) that would otherwise land on the user's
# first reply. A throwaway synth here moves it to startup. No-op
# cost on CPU.
try:
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as _wt:
_wp = _wt.name
model.tts_to_file("워밍업", speaker_id, _wp, speed=SPEED)
try:
os.unlink(_wp)
except OSError:
pass
except Exception as _we: # pragma: no cover
print(f"[melo-worker] warmup synth skipped: {_we}", flush=True)
print( print(
f"[melo-worker] ready (lang={LANGUAGE} speed={SPEED} " f"[melo-worker] ready (lang={LANGUAGE} speed={SPEED} "
f"device={DEVICE} speakers={list(spk_map.keys())})", f"device={DEVICE} speakers={list(spk_map.keys())})",

View File

@@ -52,11 +52,18 @@ from flask import Flask, request, jsonify, Response, stream_with_context
try: # package-relative when imported as ``bridge.server`` try: # package-relative when imported as ``bridge.server``
from bridge.text_utils import split_sentences from bridge.text_utils import split_sentences
from bridge.stt_filter import filter_speech_segments, has_speech from bridge.stt_filter import filter_speech_segments, has_speech
from bridge import settings_web
except ImportError: # script-relative when run as ``bridge/server.py`` except ImportError: # script-relative when run as ``bridge/server.py``
from text_utils import split_sentences from text_utils import split_sentences
from stt_filter import filter_speech_segments, has_speech from stt_filter import filter_speech_segments, has_speech
import settings_web
app = Flask(__name__) app = Flask(__name__)
# Settings web UI (/settings) — change models/language/TTS/instructions live.
try:
settings_web.register(app)
except Exception as _e: # pragma: no cover - never block the bridge on the UI
print(f"[bridge] settings UI unavailable: {_e}", flush=True)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Configuration (env-driven; see .env.example) # Configuration (env-driven; see .env.example)
@@ -83,7 +90,20 @@ STT_LANGUAGE = os.environ.get("STT_LANGUAGE", "ko").strip() or None
# TTS engine: "melo" (MeloTTS Korean speaker, the warm worker) is the primary # TTS engine: "melo" (MeloTTS Korean speaker, the warm worker) is the primary
# voice; Piper is kept as a fallback if the worker is unreachable. Set # voice; Piper is kept as a fallback if the worker is unreachable. Set
# TTS_ENGINE=piper to disable MeloTTS entirely. # TTS_ENGINE=piper to disable MeloTTS entirely.
TTS_ENGINE = os.environ.get("TTS_ENGINE", "melo").strip().lower() def _tts_engine_setting() -> str:
"""TTS engine: settings-UI value (runtime config JSON) wins, else env, else
melo. Read at startup; the settings UI restarts the bridge on apply."""
try:
_cp = os.environ.get("JARVIS_CONFIG_PATH", "/app/config/jarvis.json")
_v = json.loads(open(_cp, encoding="utf-8").read()).get("tts_engine")
if _v:
return str(_v).strip().lower()
except Exception:
pass
return os.environ.get("TTS_ENGINE", "melo").strip().lower()
TTS_ENGINE = _tts_engine_setting()
MELO_WORKER_URL = os.environ.get("MELO_WORKER_URL", "http://127.0.0.1:8770") MELO_WORKER_URL = os.environ.get("MELO_WORKER_URL", "http://127.0.0.1:8770")
MELO_TIMEOUT = float(os.environ.get("MELO_TIMEOUT", "30")) MELO_TIMEOUT = float(os.environ.get("MELO_TIMEOUT", "30"))
# When MeloTTS is the engine, do NOT silently fall back to the English Piper # When MeloTTS is the engine, do NOT silently fall back to the English Piper
@@ -459,14 +479,31 @@ def http_converse_stream():
# own Date.now() capture timestamps (same host, same clock). # own Date.now() capture timestamps (same host, same clock).
return int(time.time() * 1000) return int(time.time() * 1000)
# Length of the captured speech clip (16-bit mono PCM). This is the
# "음성 인식(녹음)" portion — how long the user actually spoke (+ the
# bot's trailing silence cutoff) — as opposed to "STT 처리", the Whisper
# transcription time below. Splitting them shows whether a slow turn is
# the listening/recording or the transcription.
try:
_frames, _sr = _read_wav_pcm(raw)
audio_sec = (len(_frames) / 2) / _sr if _sr else 0.0
except Exception:
audio_sec = 0.0
t0 = time.monotonic() t0 = time.monotonic()
stt = transcribe(raw) stt = transcribe(raw)
t_stt = time.monotonic() t_stt = time.monotonic()
transcript = stt.get("text", "") transcript = stt.get("text", "")
if not transcript: if not transcript:
print(
f"[bridge] ⏱️ turn 녹음(음성)={audio_sec:.1f}s STT처리(whisper)={t_stt - t0:.1f}s "
f"→ 인식 결과 없음 ({stt.get('note', '빈 결과')})",
flush=True,
)
yield json.dumps({"type": "meta", "transcript": "", "language": stt.get("language"), yield json.dumps({"type": "meta", "transcript": "", "language": stt.get("language"),
"reply": "", "error": stt.get("error"), "reply": "", "error": stt.get("error"),
"note": stt.get("note", "빈 결과"), "note": stt.get("note", "빈 결과"),
"audio_sec": round(audio_sec, 1),
"stt_sec": round(t_stt - t0, 1), "broadcast_action": None}) + "\n" "stt_sec": round(t_stt - t0, 1), "broadcast_action": None}) + "\n"
yield json.dumps({"type": "end"}) + "\n" yield json.dumps({"type": "end"}) + "\n"
return return
@@ -482,6 +519,7 @@ def http_converse_stream():
"reply": reply, "reply": reply,
"error": result.get("error"), "error": result.get("error"),
"note": "ok" if reply.strip() else "답변 없음", "note": "ok" if reply.strip() else "답변 없음",
"audio_sec": round(audio_sec, 1),
"stt_sec": round(t_stt - t0, 1), "stt_sec": round(t_stt - t0, 1),
"think_sec": round(t_think - t_stt, 1), "think_sec": round(t_think - t_stt, 1),
# Wall-clock LLM window (epoch ms) for the transcript-channel timing # Wall-clock LLM window (epoch ms) for the transcript-channel timing
@@ -516,8 +554,9 @@ def http_converse_stream():
"tts_end_ms": tts_end_ms, "tts_end_ms": tts_end_ms,
}) + "\n" }) + "\n"
print( print(
f"[bridge] ⏱️ turn stt={t_stt - t0:.1f}s think(LLM)={t_think - t_stt:.1f}s " f"[bridge] ⏱️ turn 녹음(음성)={audio_sec:.1f}s STT처리(whisper)={t_stt - t0:.1f}s "
f"tts={tts_total:.1f}s total={time.monotonic() - t0:.1f}s replylen={len(reply)} " f"think(LLM)={t_think - t_stt:.1f}s tts={tts_total:.1f}s "
f"total(STT~TTS)={time.monotonic() - t0:.1f}s replylen={len(reply)} "
f"transcript={transcript[:40]!r}", f"transcript={transcript[:40]!r}",
flush=True, flush=True,
) )

201
bridge/settings_web.py Normal file
View File

@@ -0,0 +1,201 @@
"""Settings web UI for the Jarvis bridge.
A small in-app page (served by the Flask bridge) to change models, language,
TTS and the LLM instructions WITHOUT editing files or rebuilding. Writes to the
runtime config JSON (JARVIS_CONFIG_PATH) that ``load_settings()`` reads, then
restarts the bridge (and TTS worker) via supervisord so changes take effect.
Internal-network use only (no auth, per deployment decision).
"""
from __future__ import annotations
import json
import os
import subprocess
import urllib.request
from pathlib import Path
from typing import Any, Dict
# Fields the UI manages. Each maps to a key in the runtime config JSON, with a
# label and an input kind for the form.
FIELDS = [
("ollama_chat_model", "LLM 모델", "model"),
("whisper_model", "STT(Whisper) 모델", "select:tiny,base,small,medium,large,large-v3"),
("tts_engine", "TTS 엔진", "select:melo,piper"),
("melo_speed", "TTS 속도 (MeloTTS)", "number:0.5:2.5:0.1"),
("output_language", "출력 언어 (비우면 사용자 언어)", "text"),
("llm_thinking_enabled", "LLM 사고(thinking) 모드", "bool"),
("agentic_max_turns", "에이전트 최대 반복", "number:1:12:1"),
("llm_instructions", "LLM 추가 지침 (시스템 프롬프트에 덧붙임)", "textarea"),
]
_KEYS = [k for k, _, _ in FIELDS]
def _config_path() -> Path:
p = os.environ.get("JARVIS_CONFIG_PATH")
return Path(p).expanduser() if p else (Path.home() / ".config" / "jarvis" / "config.json")
def _persist_path() -> Path:
"""Persistent overrides on the data volume — survive container recreate.
entrypoint.sh merges this back onto the env-rendered config at startup."""
return Path(os.environ.get("JARVIS_SETTINGS_PATH") or "/data/jarvis-settings.json")
def _read_config() -> Dict[str, Any]:
try:
return json.loads(_config_path().read_text("utf-8"))
except Exception:
return {}
def _current() -> Dict[str, Any]:
cfg = _read_config()
out: Dict[str, Any] = {}
for k in _KEYS:
if k == "melo_speed":
out[k] = cfg.get("melo_speed", os.environ.get("MELO_SPEED", "1.5"))
elif k == "output_language":
out[k] = cfg.get("output_language", os.environ.get("OUTPUT_LANGUAGE", ""))
else:
out[k] = cfg.get(k, "")
return out
def _ollama_models() -> list[str]:
base = os.environ.get("OLLAMA_BASE_URL", "http://127.0.0.1:11434").rstrip("/")
try:
with urllib.request.urlopen(f"{base}/api/tags", timeout=4) as r:
data = json.loads(r.read())
return sorted(m.get("name", "") for m in data.get("models", []) if m.get("name"))
except Exception:
return []
def _coerce(updates: Dict[str, Any]) -> Dict[str, Any]:
clean: Dict[str, Any] = {}
for k, v in updates.items():
if k not in _KEYS:
continue
if k == "melo_speed":
try:
v = float(v)
except (TypeError, ValueError):
continue
elif k == "agentic_max_turns":
try:
v = int(v)
except (TypeError, ValueError):
continue
elif k == "llm_thinking_enabled":
v = str(v).lower() in ("1", "true", "on", "yes")
clean[k] = v
return clean
def _write_merge(path: Path, clean: Dict[str, Any]) -> None:
cur: Dict[str, Any] = {}
try:
cur = json.loads(path.read_text("utf-8"))
except Exception:
cur = {}
cur.update(clean)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(cur, ensure_ascii=False, indent=2), "utf-8")
def _save(updates: Dict[str, Any]) -> None:
clean = _coerce(updates)
# 1) persistent overrides (survive `docker compose up` recreate)
_write_merge(_persist_path(), clean)
# 2) runtime config so a bridge/worker restart picks it up immediately
_write_merge(_config_path(), clean)
def _apply() -> str:
# Restart melo + bridge AFTER this response is sent. Detached (new session)
# so the bridge being killed mid-restart doesn't drop the restart itself,
# and the HTTP client still receives this response.
try:
subprocess.Popen(
["sh", "-c", "sleep 1; supervisorctl restart melo-worker bridge"],
start_new_session=True,
)
return "1초 후 브리지/TTS 워커가 재시작되어 반영됩니다."
except Exception as e: # pragma: no cover
return str(e)
_PAGE = """<!doctype html><html lang=ko><head><meta charset=utf-8>
<meta name=viewport content="width=device-width,initial-scale=1">
<title>Jarvis 설정</title><style>
body{font-family:system-ui,Segoe UI,Apple SD Gothic Neo,sans-serif;max-width:680px;margin:24px auto;padding:0 16px;color:#222}
h1{font-size:20px}label{display:block;margin:14px 0 4px;font-weight:600}
input,select,textarea{width:100%;padding:8px;border:1px solid #ccc;border-radius:8px;font-size:14px;box-sizing:border-box}
textarea{min-height:90px}.row{margin-bottom:6px}.btns{margin-top:18px;display:flex;gap:8px}
button{padding:10px 16px;border:0;border-radius:8px;font-size:14px;cursor:pointer}
.save{background:#2d6cdf;color:#fff}.apply{background:#16a34a;color:#fff}
#msg{margin-top:12px;color:#16a34a;white-space:pre-wrap}.hint{color:#888;font-weight:400;font-size:12px}
</style></head><body>
<h1>⚙️ Jarvis 설정</h1>
<p class=hint>저장 후 [적용]을 누르면 브리지/TTS가 재시작되며 반영됩니다. (내부망 전용)</p>
<form id=f></form>
<div class=btns><button class=save type=button onclick=save()>저장</button>
<button class=apply type=button onclick=apply()>저장 후 적용(재시작)</button></div>
<div id=msg></div>
<script>
const FIELDS=__FIELDS__, MODELS=__MODELS__, CUR=__CUR__;
const f=document.getElementById('f');
for(const [k,label,kind] of FIELDS){
const id='fld_'+k; let el;
if(k==='ollama_chat_model' && MODELS.length){
el=`<select id="${id}">`+MODELS.map(m=>`<option ${m===CUR[k]?'selected':''}>${m}</option>`).join('')+`</select>`;
} else if(kind.startsWith('select:')){
el='<select id="'+id+'">'+kind.slice(7).split(',').map(o=>`<option ${o===CUR[k]?'selected':''}>${o}</option>`).join('')+'</select>';
} else if(kind==='textarea'){
el=`<textarea id="${id}">${CUR[k]??''}</textarea>`;
} else if(kind==='bool'){
el=`<select id="${id}"><option value=false ${!CUR[k]?'selected':''}>off</option><option value=true ${CUR[k]?'selected':''}>on</option></select>`;
} else if(kind.startsWith('number:')){
const [mn,mx,st]=kind.slice(7).split(':');
el=`<input id="${id}" type=number min=${mn} max=${mx} step=${st} value="${CUR[k]??''}">`;
} else { el=`<input id="${id}" type=text value="${CUR[k]??''}">`; }
f.insertAdjacentHTML('beforeend',`<div class=row><label>${label}</label>${el}</div>`);
}
function collect(){const o={};for(const [k] of FIELDS){o[k]=document.getElementById('fld_'+k).value;}return o;}
async function post(url){const r=await fetch(url,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(collect())});return r.json();}
async function save(){const j=await post('/api/settings');document.getElementById('msg').textContent=j.ok?'저장됨':'오류: '+(j.error||'');}
async function apply(){await post('/api/settings');const j=await fetch('/api/settings/apply',{method:'POST'}).then(r=>r.json());document.getElementById('msg').textContent='적용: '+(j.result||j.error||'');}
</script></body></html>"""
def register(app) -> None:
"""Attach the settings routes to the Flask ``app``."""
from flask import request, jsonify, Response
@app.get("/settings")
def _settings_page(): # noqa: ANN202
html = (
_PAGE.replace("__FIELDS__", json.dumps(FIELDS, ensure_ascii=False))
.replace("__MODELS__", json.dumps(_ollama_models()))
.replace("__CUR__", json.dumps(_current(), ensure_ascii=False))
)
return Response(html, mimetype="text/html")
@app.get("/api/settings")
def _get_settings(): # noqa: ANN202
return jsonify({"ok": True, "settings": _current(), "models": _ollama_models()})
@app.post("/api/settings")
def _post_settings(): # noqa: ANN202
data = request.get_json(silent=True) or {}
try:
_save(data)
return jsonify({"ok": True})
except Exception as e: # pragma: no cover
return jsonify({"ok": False, "error": str(e)}), 500
@app.post("/api/settings/apply")
def _apply_settings(): # noqa: ANN202
return jsonify({"ok": True, "result": _apply()})

View File

@@ -0,0 +1,14 @@
# GPU override for LINUX hosts using nvidia-container-toolkit with CDI
# (Ubuntu local Docker). Verified on the RTX 5050 (Blackwell sm_120).
#
# docker compose -f docker-compose.yml -f docker-compose.gpu-linux.yml up -d
#
# Or set COMPOSE_FILE in .env (recommended):
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
services:
ollama:
devices:
- "nvidia.com/gpu=all"
javis:
devices:
- "nvidia.com/gpu=all"

View File

@@ -0,0 +1,26 @@
# GPU override for WINDOWS 11 (Docker Desktop + WSL2 + NVIDIA) and any host
# that exposes the GPU through Docker's portable device-reservation API rather
# than CDI. Requires the NVIDIA GPU driver on Windows and GPU support enabled in
# Docker Desktop (Settings → Resources → WSL Integration / GPU).
#
# docker compose -f docker-compose.yml -f docker-compose.gpu-windows.yml up -d
#
# Or set COMPOSE_FILE in .env (recommended):
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml
services:
ollama:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
javis:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]

View File

@@ -27,10 +27,9 @@ services:
# model resident forever, wasting VRAM next to the chat model. # model resident forever, wasting VRAM next to the chat model.
volumes: volumes:
- ollama_models:/root/.ollama - ollama_models:/root/.ollama
# GPU: needs nvidia-container-toolkit on the host (CDI). Verified on the # GPU is added by a platform override (see docker-compose.gpu-linux.yml /
# RTX 5050 (Blackwell sm_120) — Ollama offloads 100% to GPU. # docker-compose.gpu-windows.yml + COMPOSE_FILE in .env). Base stays
devices: # GPU-agnostic so the same files run on Ubuntu (CDI) and Windows (Desktop).
- "nvidia.com/gpu=all"
# Auto-pull the models the brain needs, then exit. Idempotent (re-runnable). # Auto-pull the models the brain needs, then exit. Idempotent (re-runnable).
ollama-init: ollama-init:
@@ -67,20 +66,43 @@ services:
WHISPER_MODEL: ${WHISPER_MODEL:-medium} WHISPER_MODEL: ${WHISPER_MODEL:-medium}
WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda} WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16} WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
# MeloTTS on the GPU (cu128 torch baked by docker/setup-melo.sh). CPU synth
# serialised under load and pushed TTS to 7-8s; GPU does ~0.3s/sentence.
MELO_DEVICE: ${MELO_DEVICE:-cuda}
# Optional single-language lock for replies (empty = user's own language). # Optional single-language lock for replies (empty = user's own language).
OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-} OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-ko}
# Drop the pre-loop planner LLM call to cut voice-reply latency on small # Drop the pre-loop planner LLM call to cut voice-reply latency on small
# hardware (the planner adds a full model round-trip per turn). # hardware (the planner adds a full model round-trip per turn).
PLANNER_ENABLED: ${PLANNER_ENABLED:-0} PLANNER_ENABLED: ${PLANNER_ENABLED:-0}
# Lock STT to Korean (skip Whisper auto-detect). # Lock STT to Korean (skip Whisper auto-detect).
STT_LANGUAGE: ${STT_LANGUAGE:-ko} STT_LANGUAGE: ${STT_LANGUAGE:-ko}
VOICE_SILENCE_MS: ${VOICE_SILENCE_MS:-600}
BRIDGE_URL: http://127.0.0.1:8765 BRIDGE_URL: http://127.0.0.1:8765
depends_on: # Split-deployment role: full (default, all-in-one), browser (only the
- ollama # desktop + Chrome + CDP, reused over the LAN), or bot (only bot + bridge
# GPU: accelerates Whisper STT (and anything else CUDA) in this container. # + TTS, driving a remote browser via CDP_HOST). See docker/run-if-role.sh.
# Verified: faster-whisper float16 works on the RTX 5050 (sm_120). JARVIS_ROLE: ${JARVIS_ROLE:-full}
devices: # Chrome CDP bind address INSIDE the container. 0.0.0.0 lets a remote bot
- "nvidia.com/gpu=all" # (JARVIS_ROLE=bot on another PC) drive this host's browser. Loopback by
# default so the all-in-one layout stays unreachable off-host.
CDP_BIND: ${CDP_BIND:-127.0.0.1}
CDP_PORT: ${CDP_PORT:-9222}
# Where the bot drives Chrome. Loopback for full/browser; on a remote bot
# set CDP_HOST to the browser host's LAN IP (e.g. 192.168.10.9).
CDP_HOST: ${CDP_HOST:-127.0.0.1}
# Browser-control endpoint. The browser host serves it (BIND/PORT); a
# remote bot sets BROWSER_CONTROL_URL=http://<browser-host>:8777 so its
# controlBrowser tool posts there instead of running node locally. Empty
# on full/browser → the tool runs chrome-control.mjs locally.
BROWSER_CONTROL_BIND: ${BROWSER_CONTROL_BIND:-0.0.0.0}
BROWSER_CONTROL_PORT: ${BROWSER_CONTROL_PORT:-8777}
BROWSER_CONTROL_URL: ${BROWSER_CONTROL_URL:-}
# No hard depends_on ollama: a browser-host (`docker compose up -d javis`)
# must NOT pull in Ollama. Full/bot layouts start it with a plain
# `docker compose up -d` (all services); the bridge tolerates Ollama warming
# up lazily, so start order doesn't matter.
# GPU is added by a platform override (docker-compose.gpu-linux.yml /
# docker-compose.gpu-windows.yml). The browser-only host needs no GPU.
shm_size: "1gb" # Chrome needs a larger /dev/shm shm_size: "1gb" # Chrome needs a larger /dev/shm
ports: ports:
# All published to loopback only by default — VNC/noVNC use a weak default # All published to loopback only by default — VNC/noVNC use a weak default
@@ -91,6 +113,15 @@ services:
# .env pins VNC_PORT=5902. # .env pins VNC_PORT=5902.
- "${VNC_BIND:-127.0.0.1}:${VNC_PORT:-5901}:5901" # VNC - "${VNC_BIND:-127.0.0.1}:${VNC_PORT:-5901}:5901" # VNC
- "${VNC_BIND:-127.0.0.1}:${NOVNC_PORT:-6080}:6080" # noVNC (browser) - "${VNC_BIND:-127.0.0.1}:${NOVNC_PORT:-6080}:6080" # noVNC (browser)
# Chrome CDP for a remote bot (JARVIS_ROLE=bot). Loopback by default; for a
# LAN browser-host set CDP_PUBLISH_BIND=0.0.0.0 (internal network, no auth).
- "${CDP_PUBLISH_BIND:-127.0.0.1}:${CDP_PORT:-9222}:9222" # Chrome CDP
# Browser-control endpoint a remote bot posts to (real xdotool input runs
# on THIS host). Published on the LAN for the browser-host layout.
- "${CDP_PUBLISH_BIND:-127.0.0.1}:${BROWSER_CONTROL_PORT:-8777}:8777" # control-server
# Settings UI + brain API (bridge). Reach it at http://localhost:8765/settings
# on the bot host. Requires BRIDGE_HOST=0.0.0.0 (set in .env) to forward.
- "${SETTINGS_PUBLISH_BIND:-127.0.0.1}:${BRIDGE_PORT:-8765}:8765" # bridge / settings
# The brain bridge is NOT published: it binds the container's loopback # The brain bridge is NOT published: it binds the container's loopback
# (BRIDGE_HOST=127.0.0.1) and is only consumed by the bot in this same # (BRIDGE_HOST=127.0.0.1) and is only consumed by the bot in this same
# container, so it needs no host port and stays unreachable off-container. # container, so it needs no host port and stays unreachable off-container.

View File

@@ -47,6 +47,22 @@ chmod 600 /root/.vnc/passwd
# --- Render jarvis brain config from template --- # --- Render jarvis brain config from template ---
envsubst < /app/docker/jarvis-config.template.json > /app/config/jarvis.json envsubst < /app/docker/jarvis-config.template.json > /app/config/jarvis.json
export JARVIS_CONFIG_PATH=/app/config/jarvis.json export JARVIS_CONFIG_PATH=/app/config/jarvis.json
# Merge persistent settings from the settings UI (on the /data volume) on top of
# the env-rendered config, so changes survive container recreate.
if [ -f /data/jarvis-settings.json ]; then
python3 - <<'PY' || true
import json
try:
base = json.load(open("/app/config/jarvis.json"))
ov = json.load(open("/data/jarvis-settings.json"))
if isinstance(base, dict) and isinstance(ov, dict):
base.update(ov)
json.dump(base, open("/app/config/jarvis.json", "w"), ensure_ascii=False, indent=2)
print("[entrypoint] merged persistent settings overrides")
except Exception as e:
print(f"[entrypoint] settings merge skipped: {e}")
PY
fi
# --- Ensure the Piper voice exists (best effort) --- # --- Ensure the Piper voice exists (best effort) ---
bash /app/docker/download-piper.sh || echo "[entrypoint] piper download failed; TTS may be unavailable" bash /app/docker/download-piper.sh || echo "[entrypoint] piper download failed; TTS may be unavailable"

View File

@@ -8,13 +8,26 @@ for i in $(seq 1 40); do
done done
sleep 3 sleep 3
export DISPLAY=:1 export DISPLAY=:1
# --remote-debugging-port exposes CDP so the brain's browse-search.mjs
# (playwright connectOverCDP) can drive this on-screen Chrome for the # Suppress the "--no-sandbox unsupported flag" warning bar via a managed policy
# broadcast-visible Google/YouTube search. Bound to loopback (same container). # instead of --test-type. --test-type is an automation signal Google can flag,
# so we keep the launch flags minimal/clean (less chance of the /sorry/ bot
# challenge) while still hiding the infobar.
mkdir -p /etc/opt/chrome/policies/managed
cat > /etc/opt/chrome/policies/managed/jarvis.json <<'JSON'
{ "CommandLineFlagSecurityWarningsEnabled": false }
JSON
# Minimal, non-automation flags. --remote-debugging exposes CDP so the brain can
# drive this on-screen Chrome (Google/YouTube/Naver), --disable-features=Translate
# hides the translate popup. NO --test-type / --disable-blink-features.
exec google-chrome \ exec google-chrome \
--no-sandbox --no-first-run --disable-dev-shm-usage \ --no-sandbox --no-first-run --disable-dev-shm-usage \
--no-default-browser-check \
--disable-features=Translate,TranslateUI \
--lang=ko-KR \
--remote-debugging-port="${CDP_PORT:-9222}" \ --remote-debugging-port="${CDP_PORT:-9222}" \
--remote-debugging-address=127.0.0.1 \ --remote-debugging-address="${CDP_BIND:-127.0.0.1}" \
--user-data-dir="${CHROME_PROFILE_DIR:-/root/chrome-profile}" \ --user-data-dir="${CHROME_PROFILE_DIR:-/root/chrome-profile}" \
--password-store=basic --start-maximized \ --password-store=basic --start-maximized \
"${CHROME_START_URL:-about:blank}" "${CHROME_START_URL:-about:blank}"

22
docker/run-if-role.sh Executable file
View File

@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Role guard for split deployments.
#
# run-if-role.sh <roles-csv> <command...>
#
# Runs <command> only when JARVIS_ROLE is one of <roles-csv> (or "full"/unset).
# Otherwise it idles so supervisord keeps the program slot "running" without
# doing any work. This lets ONE image serve three layouts:
#
# JARVIS_ROLE=full (default) everything in one container
# JARVIS_ROLE=browser only the desktop + Chrome + CDP (reused over the LAN)
# JARVIS_ROLE=bot only the bot + bridge + TTS (drives a remote browser
# via CDP_HOST/CDP_PORT)
set -e
want="$1"; shift
role="${JARVIS_ROLE:-full}"
if [ "$role" = "full" ]; then exec "$@"; fi
case ",$want," in
*",$role,"*) exec "$@" ;;
esac
echo "[role-guard] JARVIS_ROLE=$role not in '$want' — idling: $*" >&2
exec sleep infinity

View File

@@ -9,8 +9,11 @@
# - It isolates the heavy torch/transformers stack from the slim bridge env, # - It isolates the heavy torch/transformers stack from the slim bridge env,
# which pins numpy<2 for faster-whisper. # which pins numpy<2 for faster-whisper.
# #
# torch is pinned to the CPU build: TTS runs on CPU so the GPU stays reserved # torch is the CUDA (cu128) build so MeloTTS runs on the GPU alongside Ollama +
# for Ollama + Whisper, and we avoid pulling multi-GB CUDA wheels. # Whisper. CPU synth serialised under concurrent load (whisper STT + bot) and
# blew TTS up to 7-8s per reply; on the GPU a sentence synthesises in ~0.3s.
# cu128 is the Blackwell (sm_120) wheel verified on this host's RTX 5050.
# The worker selects the device via MELO_DEVICE=cuda (compose).
# ============================================================================ # ============================================================================
set -euxo pipefail set -euxo pipefail
@@ -29,11 +32,11 @@ rm -rf /var/lib/apt/lists/*
python3.11 -m venv /opt/melo python3.11 -m venv /opt/melo
/opt/melo/bin/pip install --no-cache-dir --upgrade pip wheel setuptools /opt/melo/bin/pip install --no-cache-dir --upgrade pip wheel setuptools
# CPU-only torch first, so MeloTTS's unpinned `torch` dep is already satisfied # CUDA (cu128) torch first, so MeloTTS's unpinned `torch` dep is already
# and pip does not pull the CUDA build. Pinned for reproducible rebuilds (these # satisfied with the GPU build. Pinned to the Blackwell-verified versions
# are the versions the CPU index resolved when this layer was verified). # (2.11.0+cu128) for reproducible rebuilds.
/opt/melo/bin/pip install --no-cache-dir torch==2.12.0 torchaudio==2.11.0 \ /opt/melo/bin/pip install --no-cache-dir torch==2.11.0+cu128 torchaudio==2.11.0+cu128 \
--index-url https://download.pytorch.org/whl/cpu --index-url https://download.pytorch.org/whl/cu128
# MeloTTS from GitHub. The PyPI sdist is broken (its setup.py reads a # MeloTTS from GitHub. The PyPI sdist is broken (its setup.py reads a
# requirements.txt that is not shipped in the sdist), so install from the repo. # requirements.txt that is not shipped in the sdist), so install from the repo.

View File

@@ -14,7 +14,7 @@ serverurl=unix:///run/supervisor.sock
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[program:xvnc] [program:xvnc]
command=/app/docker/run-xvnc.sh command=/app/docker/run-if-role.sh full,browser /app/docker/run-xvnc.sh
priority=100 priority=100
autorestart=true autorestart=true
stdout_logfile=/dev/stdout stdout_logfile=/dev/stdout
@@ -23,7 +23,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 stderr_logfile_maxbytes=0
[program:pulse] [program:pulse]
command=/app/docker/run-pulse.sh command=/app/docker/run-if-role.sh full,browser /app/docker/run-pulse.sh
priority=150 priority=150
autorestart=true autorestart=true
stdout_logfile=/dev/stdout stdout_logfile=/dev/stdout
@@ -32,7 +32,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 stderr_logfile_maxbytes=0
[program:xfce] [program:xfce]
command=/app/docker/run-xfce.sh command=/app/docker/run-if-role.sh full,browser /app/docker/run-xfce.sh
priority=200 priority=200
autorestart=true autorestart=true
stdout_logfile=/dev/stdout stdout_logfile=/dev/stdout
@@ -41,7 +41,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 stderr_logfile_maxbytes=0
[program:novnc] [program:novnc]
command=websockify --web=/usr/share/novnc 6080 localhost:5901 command=/app/docker/run-if-role.sh full,browser websockify --web=/usr/share/novnc 6080 localhost:5901
priority=250 priority=250
autorestart=true autorestart=true
stdout_logfile=/dev/stdout stdout_logfile=/dev/stdout
@@ -52,7 +52,7 @@ stderr_logfile_maxbytes=0
[program:melo-worker] [program:melo-worker]
; Warm MeloTTS Korean voice (speed 1.5) in its own py3.11 venv. The bridge's ; Warm MeloTTS Korean voice (speed 1.5) in its own py3.11 venv. The bridge's
; synthesize() POSTs here; if this is down the bridge falls back to Piper. ; synthesize() POSTs here; if this is down the bridge falls back to Piper.
command=/opt/melo/bin/python /app/bridge/melo_worker.py command=/app/docker/run-if-role.sh full,bot /opt/melo/bin/python /app/bridge/melo_worker.py
directory=/app directory=/app
; HF_HOME points at the dedicated, image-baked melo cache (warmed in ; HF_HOME points at the dedicated, image-baked melo cache (warmed in
; setup-melo.sh). The brain's whisper_cache volume is mounted over ; setup-melo.sh). The brain's whisper_cache volume is mounted over
@@ -61,7 +61,10 @@ directory=/app
; HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE force pure-cache reads: the pinned old ; HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE force pure-cache reads: the pinned old
; transformers/huggingface_hub otherwise retry the network on every load and ; transformers/huggingface_hub otherwise retry the network on every load and
; error out instead of falling back to the (complete) baked cache. ; error out instead of falling back to the (complete) baked cache.
environment=MELO_LANGUAGE="KR",MELO_SPEED="1.5",MELO_DEVICE="cpu",MELO_WORKER_HOST="127.0.0.1",MELO_WORKER_PORT="8770",HF_HOME="/opt/melo-cache",HF_HUB_OFFLINE="1",TRANSFORMERS_OFFLINE="1" ; MELO_DEVICE inherits from the container env (compose sets it; default cuda)
; so the worker runs MeloTTS on the GPU. supervisord interpolates %(ENV_x)s
; from its own environment, which is the container's.
environment=MELO_LANGUAGE="KR",MELO_SPEED="1.5",MELO_DEVICE="%(ENV_MELO_DEVICE)s",MELO_WORKER_HOST="127.0.0.1",MELO_WORKER_PORT="8770",HF_HOME="/opt/melo-cache",HF_HUB_OFFLINE="1",TRANSFORMERS_OFFLINE="1"
priority=280 priority=280
autorestart=true autorestart=true
stdout_logfile=/dev/stdout stdout_logfile=/dev/stdout
@@ -70,7 +73,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 stderr_logfile_maxbytes=0
[program:bridge] [program:bridge]
command=/opt/venv/bin/python -m bridge.server command=/app/docker/run-if-role.sh full,bot /opt/venv/bin/python -m bridge.server
directory=/app directory=/app
priority=300 priority=300
autorestart=true autorestart=true
@@ -80,7 +83,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 stderr_logfile_maxbytes=0
[program:chrome] [program:chrome]
command=/app/docker/run-chrome.sh command=/app/docker/run-if-role.sh full,browser /app/docker/run-chrome.sh
priority=350 priority=350
autorestart=true autorestart=true
stdout_logfile=/dev/stdout stdout_logfile=/dev/stdout
@@ -88,8 +91,21 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 stderr_logfile_maxbytes=0
[program:control-server]
; Browser-control HTTP endpoint on the BROWSER HOST. A remote `bot` posts
; commands here so xdotool / CDP run on THIS machine (real input on this
; screen). Only meaningful in full/browser roles. Internal network only.
command=/app/docker/run-if-role.sh full,browser node /app/bot/scripts/stream-test/control-server.mjs
directory=/app/bot
priority=360
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:bot] [program:bot]
command=/app/docker/run-bot.sh command=/app/docker/run-if-role.sh full,bot /app/docker/run-bot.sh
directory=/app/bot directory=/app/bot
priority=400 priority=400
autorestart=true autorestart=true

73
docs/DEPLOY.md Normal file
View File

@@ -0,0 +1,73 @@
# Deployment layouts
One image, three roles (`JARVIS_ROLE`), selected in `.env`. GPU is added per OS
via a compose override picked with `COMPOSE_FILE`.
## A. All-in-one (single machine)
Everything (desktop + Chrome + bridge + bot + TTS) in one container.
```
# .env
JARVIS_ROLE=full
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml # Windows 11
DISCORD_SELFBOT_TOKEN=...
DISCORD_GUILD_ID=...
docker compose up -d # Ollama + javis (COMPOSE_FILE adds GPU)
```
## B. Split: browser host (LAN) + bot on your PC
The on-screen Chrome, real mouse/keyboard (xdotool) and screen live on the
**browser host**. Your PC runs the **bot** and drives that browser over the
internal network — no auth (internal only).
### Browser host (the LAN machine that shows Chrome, e.g. 192.168.10.9)
```
# .env
JARVIS_ROLE=browser
CDP_BIND=0.0.0.0
BROWSER_CONTROL_BIND=0.0.0.0
CDP_PUBLISH_BIND=0.0.0.0
# no GPU needed → leave COMPOSE_FILE unset (base compose only)
docker compose up -d javis # desktop + Chrome + control-server (port 8777)
```
Watch it on this machines VNC (`localhost:5901`) / noVNC (`localhost:6080`).
### Bot host (your PC — Ubuntu or Windows 11)
```
# .env
JARVIS_ROLE=bot
BROWSER_CONTROL_URL=http://192.168.10.9:8777 # the browser host's LAN IP
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu
# COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-windows.yml # Windows 11
DISCORD_SELFBOT_TOKEN=...
DISCORD_GUILD_ID=...
docker compose up -d # bot + bridge + TTS + Ollama (GPU per OS)
```
The bots `controlBrowser` tool posts commands to `BROWSER_CONTROL_URL`, so
"네이버에서 X 검색", "구글로 돌아가" etc. drive the **browser hosts** Chrome with real
human-style input (visible on its VNC).
## Windows 11 notes
- Install the NVIDIA driver on Windows and enable GPU in Docker Desktop
(Settings → Resources → WSL Integration). Use the `gpu-windows.yml` override.
- Paths: named volumes are cross-platform. The Gemini OAuth bind mount defaults
to `${HOME}/.config/javis/gemini` (works under WSL); override `GEMINI_OAUTH_DIR`
if needed.
## Known limitation
Discord Go-Live broadcast of the **browser host's** screen from a **remote** bot
is not supported (the bot's WebRTC screen capture is local to the bot machine).
Use the browser host's VNC to view it. A full remote-broadcast path is separate,
larger work.

View File

@@ -12,7 +12,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
- **Inputs**: - **Inputs**:
- Redacted user query - Redacted user query
- Recent dialogue (last 5 minutes), including in-loop tool-call + tool-role messages from prior replies within the active conversation (tool carryover, `DialogueMemory.record_tool_turn` / `get_recent_turns_with_tools` in [src/jarvis/memory/conversation.py](src/jarvis/memory/conversation.py); per-prompt cap via `cfg.tool_carryover_max_turns` / `tool_carryover_per_entry_chars`; storage cap `_tool_turns_max_storage = 16`; cleared on `stop` signal AND on new-conversation entry; UNTRUSTED WEB EXTRACT fence markers preserved on truncation; both `content` and `tool_calls[*].function.arguments` scrubbed on write) - Recent dialogue (last 5 minutes), including in-loop tool-call + tool-role messages from prior replies within the active conversation (tool carryover, `DialogueMemory.record_tool_turn` / `get_recent_turns_with_tools` in [src/jarvis/memory/conversation.py](src/jarvis/memory/conversation.py); per-prompt cap via `cfg.tool_carryover_max_turns` / `tool_carryover_per_entry_chars`; storage cap `_tool_turns_max_storage = 16`; cleared on `stop` signal AND on new-conversation entry; UNTRUSTED WEB EXTRACT fence markers preserved on truncation; both `content` and `tool_calls[*].function.arguments` scrubbed on write)
- Unified system prompt from [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) + ASR note + tool-protocol guidance. Reply language is resolved by `reply_language_directive(OUTPUT_LANGUAGE, cfg.tts_engine)`: an explicit `OUTPUT_LANGUAGE` env lock wins (forces "reply only in `<language>`", also forbidding other scripts so small models stop leaking trailing CJK/Hanja); else a Piper/Chatterbox TTS forces English (English-only voices); else (multilingual TTS, no lock) the assistant replies in the user's own language. The directive is inserted near the FRONT of the guidance list so a small model gives it primacy, and when the lock is set `build_system_prompt()` also rewrites the persona's "in the user's language" clause to the locked language so the persona does not contradict the lock. Gated in `_build_initial_system_message()` at [engine.py](src/jarvis/reply/engine.py). - Unified system prompt from [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) + ASR note + tool-protocol guidance. Reply language is resolved by `reply_language_directive(lang, cfg.tts_engine)` where `lang = _resolve_output_language()` — the single source of truth that prefers the settings-web UI value (config JSON `output_language`) over the compose `OUTPUT_LANGUAGE` env, so changing the language in the settings page takes effect. An explicit lock wins (forces "reply only in `<language>`", also forbidding other scripts so small models stop leaking trailing CJK/Hanja); else a Piper/Chatterbox TTS forces English (English-only voices); else (multilingual TTS, no lock) the assistant replies in the user's own language. The directive is inserted near the FRONT of the guidance list so a small model gives it primacy, and the SAME resolved `lang` feeds `build_system_prompt()`, which rewrites the persona's "in the user's language" clause to the locked language so the persona cannot contradict the directive (previously the persona read the raw env while the directive read the config value, so a settings-UI change was honoured by one and ignored by the other). Gated in `_build_initial_system_message()` at [engine.py](src/jarvis/reply/engine.py).
- **Warm profile block** (query-agnostic User + Directives excerpt from the knowledge graph, composed by `build_warm_profile()` / `format_warm_profile_block()` in [src/jarvis/memory/graph_ops.py](src/jarvis/memory/graph_ops.py) at Step 3.5 of `reply()`; no LLM call, pure SQLite read; injected unconditionally so personalisation is the default; result cached in `DialogueMemory._hot_cache` under `DialogueMemory.WARM_PROFILE_CACHE_KEY` for the lifetime of the active conversation. Invalidated on `stop`, on new-conversation entry, AND on User/Directives graph mutations via the listener registered in [src/jarvis/daemon.py](src/jarvis/daemon.py) against `register_graph_mutation_listener` in [src/jarvis/memory/graph.py](src/jarvis/memory/graph.py); World-branch writes are ignored) - **Warm profile block** (query-agnostic User + Directives excerpt from the knowledge graph, composed by `build_warm_profile()` / `format_warm_profile_block()` in [src/jarvis/memory/graph_ops.py](src/jarvis/memory/graph_ops.py) at Step 3.5 of `reply()`; no LLM call, pure SQLite read; injected unconditionally so personalisation is the default; result cached in `DialogueMemory._hot_cache` under `DialogueMemory.WARM_PROFILE_CACHE_KEY` for the lifetime of the active conversation. Invalidated on `stop`, on new-conversation entry, AND on User/Directives graph mutations via the listener registered in [src/jarvis/daemon.py](src/jarvis/daemon.py) against `register_graph_mutation_listener` in [src/jarvis/memory/graph.py](src/jarvis/memory/graph.py); World-branch writes are ignored)
- Digested memory enrichment (optional, see #4) - Digested memory enrichment (optional, see #4)
- Time + location context (re-injected each turn) - Time + location context (re-injected each turn)

View File

@@ -825,6 +825,156 @@ def _build_enrichment_context_hint(cfg, recent_messages: list) -> Optional[str]:
return "\n\n".join(parts) if parts else None return "\n\n".join(parts) if parts else None
# Site tokens (proper nouns, not language patterns) → controlBrowser search site.
def _extra_config(key: str, default=""):
"""Read a key from the runtime config JSON (JARVIS_CONFIG_PATH) for settings
the settings-web UI manages but that aren't on the Settings dataclass
(llm_instructions, output_language override). Cheap + fail-open."""
try:
import json as _json
from pathlib import Path as _Path
p = os.environ.get("JARVIS_CONFIG_PATH")
path = _Path(p).expanduser() if p else (_Path.home() / ".config" / "jarvis" / "config.json")
return _json.loads(path.read_text("utf-8")).get(key, default) or default
except Exception:
return default
def _resolve_output_language() -> Optional[str]:
"""Single source of truth for the locked reply language.
Precedence: the settings-web UI value (config JSON) wins over the compose
``OUTPUT_LANGUAGE`` env so changing the language in the settings page takes
effect. Returns None/empty when neither is set (multilingual default).
Both the persona prompt and the reply-language directive MUST read from
here. Resolving the two independently let the persona use the env var while
the directive used the config value, so a settings-UI change rewrote the
reply directive but left the persona contradicting it.
"""
return _extra_config("output_language", "") or os.environ.get("OUTPUT_LANGUAGE")
_SITE_TOKEN_MAP = {
"네이버": "naver", "naver": "naver",
"구글": "google", "google": "google",
"유튜브": "youtube", "유투브": "youtube", "youtube": "youtube",
"다음": "daum", "daum": "daum",
"": "bing", "bing": "bing",
}
# Site homepages for the navigate (go-to / go-back) intent.
_SITE_HOME = {
"naver": "naver.com", "google": "google.com", "daum": "daum.net",
"youtube": "youtube.com", "bing": "bing.com",
}
# SEARCH intent (run a query on the site) vs NAV intent (just open / go back to
# the site). Explicit word lists because this is a DETERMINISTIC fast-path — the
# chat model narrates ("돌아갑니다") without emitting the controlBrowser call, so
# we act directly. "돌아가" (go back) is NAV, "검색" is SEARCH.
_SEARCH_WORDS = ("검색", "찾아", "search", "look up", "find")
_NAV_WORDS = (
"돌아가", "돌아와", "이동", "가줘", "가자", "열어", "들어가", "띄워", "보여",
"메인", "홈페이지", "홈으로", "back to", "go back", "go to", "open", "navigate",
)
_ALL_INTENT_WORDS = _SEARCH_WORDS + _NAV_WORDS + (
"검색해줘", "검색해", "찾아줘", "찾아봐", "열어줘", "들어가줘", "띄워줘", "보여줘",
)
def _maybe_deterministic_site_search(text: str, db, cfg, language) -> Optional[str]:
"""When broadcasting AND the user names a site AND asks to search or open/go
to it, drive the on-screen browser directly (search or navigate) so it
actually happens — the chat model only narrates ("돌아갑니다") without acting.
Fail-open: any problem returns None and the normal reply flow continues.
"""
try:
from . import turn_state
if not getattr(cfg, "stream_browser", True):
return None
if not turn_state.get_broadcasting():
return None
low = (text or "").lower()
site = tok = None
for _t, _key in _SITE_TOKEN_MAP.items():
if _t in low:
site, tok = _key, _t
break
has_search = any(w in low for w in _SEARCH_WORDS)
has_nav = any(w in low for w in _NAV_WORDS)
if not site or not (has_search or has_nav):
return None
import re
q = re.sub(re.escape(tok) + r"(에서|에다가|에다|에|로|를|을|으로)?", " ", text, flags=re.IGNORECASE)
for w in sorted(_ALL_INTENT_WORDS, key=len, reverse=True):
q = re.sub(re.escape(w), " ", q, flags=re.IGNORECASE)
q = re.sub(r"\s+", " ", q).strip(" .,!?。")
from ..tools.registry import run_tool_with_retries
if has_search and len(q) >= 2:
args = {"action": "search", "site": site, "query": q}
else:
# NAV (go back / open) — go to the site's homepage.
args = {"action": "navigate", "url": _SITE_HOME.get(site, site)}
res = run_tool_with_retries(
db=db, cfg=cfg, tool_name="controlBrowser", tool_args=args,
system_prompt="", original_prompt="", redacted_text=redact(text),
max_retries=1, language=language,
)
if res and getattr(res, "success", False):
debug_log(f"deterministic browser: {args}", "tools")
if args["action"] == "navigate":
# Don't echo the tool's mid-load url (often about:blank); give a
# clean confirmation by site name.
return f"{site} 메인 페이지로 이동했습니다."
return res.reply_text or f"{site}에서 '{q}'를 검색해 화면에 띄웠습니다."
except Exception as e: # noqa: BLE001
debug_log(f"deterministic browser failed (fail-open): {e}", "tools")
return None
_WEATHER_INTENT_WORDS = (
"날씨", "기온", "더워", "더운", "추워", "추운", "비 와", "비와", "비 올",
"눈 와", "눈와", "weather", "temperature", "forecast",
)
def _maybe_deterministic_weather(text: str, db, cfg, language) -> Optional[str]:
"""Run getWeather directly and return its concise Korean sentence, bypassing
the chat model. The 7B otherwise re-synthesises the weather into multiple
sentences and leaks units ("25도 Celsius"); the tool already formats one
clean Korean sentence, so for a plain weather ask we just return it.
Fail-open: any problem returns None and the normal flow continues.
"""
try:
low = (text or "").lower()
if not any(w in low for w in _WEATHER_INTENT_WORDS):
return None
# Extract a city candidate from the utterance (GeoIP auto-detect is
# unavailable in the container, so a named city must be passed through).
import re
_loc = text
for w in _WEATHER_INTENT_WORDS + (
"알려줘", "어때", "어떄", "말해줘", "확인해줘", "확인", "해줘",
"오늘", "지금", "현재", "", "그래서", "그럼",
):
_loc = re.sub(re.escape(w), " ", _loc, flags=re.IGNORECASE)
_loc = re.sub(r"(은|는|이|가|의|에|에서|로|을|를)\b", " ", _loc)
_loc = re.sub(r"\s+", " ", _loc).strip(" .,!?。")
args = {"location": _loc} if 1 <= len(_loc) <= 12 else {}
from ..tools.registry import run_tool_with_retries
res = run_tool_with_retries(
db=db, cfg=cfg, tool_name="getWeather", tool_args=args,
system_prompt="", original_prompt="", redacted_text=redact(text),
max_retries=1, language=language,
)
if res and getattr(res, "success", False) and res.reply_text:
debug_log("deterministic weather executed", "tools")
return res.reply_text
except Exception as e: # noqa: BLE001
debug_log(f"deterministic weather failed (fail-open): {e}", "tools")
return None
def run_reply_engine(db: "Database", cfg, tts: Optional[Any], def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
text: str, dialogue_memory: "DialogueMemory", text: str, dialogue_memory: "DialogueMemory",
language: Optional[str] = None) -> Optional[str]: language: Optional[str] = None) -> Optional[str]:
@@ -849,6 +999,20 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# Step 1: Redact sensitive information # Step 1: Redact sensitive information
redacted = redact(text) redacted = redact(text)
# Step 0.5: Deterministic on-screen site search. If the user named a site and
# asked to search/open it while broadcasting, do it directly — the small chat
# model otherwise just narrates without calling the browser tool.
_site_search_reply = _maybe_deterministic_site_search(text, db, cfg, language)
if _site_search_reply is not None:
return _site_search_reply
# Step 0.6: Deterministic weather — return getWeather's concise Korean
# sentence directly so the chat model can't rephrase it into multiple
# sentences or leak units.
_weather_reply = _maybe_deterministic_weather(text, db, cfg, language)
if _weather_reply is not None:
return _weather_reply
# Step 2: Check for recent dialogue context # Step 2: Check for recent dialogue context
recent_messages = [] recent_messages = []
is_new_conversation = True is_new_conversation = True
@@ -1027,6 +1191,19 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
"planning", "planning",
) )
# Conversational fast-path signal: did the router pick any tool that needs
# EXTERNAL DATA? Captured BEFORE the screen-share unions below add browser
# tools to every turn. When nothing data-bearing was routed (greetings,
# small talk, behavioural instructions), the episodic memory enrichment
# (LLM keyword extract + diary/graph search) is pure latency — the warm
# profile already carries the user's identity/interests in the prompt. Used
# at the needs_memory gate to skip enrichment for those turns.
_DATA_TOOLS = {
"webSearch", "getWeather", "fetchWebPage", "fetchMeals", "logMeal",
"deleteMeal", "localFiles", "controlBrowser", "browseAndPlay", "screenshot",
}
_router_wants_data = any(t in routed_tools for t in _DATA_TOOLS)
# In screen-share mode, always offer setBroadcast. "Turn the broadcast # In screen-share mode, always offer setBroadcast. "Turn the broadcast
# on/off" is language-agnostic intent the embedding/keyword router won't # on/off" is language-agnostic intent the embedding/keyword router won't
# reliably surface for a non-English utterance (e.g. "방송 꺼줘"), so the # reliably surface for a non-English utterance (e.g. "방송 꺼줘"), so the
@@ -1036,6 +1213,29 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
and "setBroadcast" not in routed_tools: and "setBroadcast" not in routed_tools:
routed_tools = routed_tools + ["setBroadcast"] routed_tools = routed_tools + ["setBroadcast"]
# In screen-share mode, always offer the on-screen browser control too. The
# small router reflexively picks webSearch for any "search/open/find" intent
# and never surfaces controlBrowser, so the model never gets the option to
# actually drive the visible browser (e.g. "네이버에서 X 검색해줘"). Offer it
# every turn; it self-gates (no-op when nothing is asked of the browser).
if getattr(cfg, "stream_browser", True):
for _bt in ("controlBrowser", "browseAndPlay"):
if _bt in _full_catalog_names and _bt not in routed_tools:
routed_tools = routed_tools + [_bt]
# When the user explicitly names a website (a proper noun, not a language
# pattern), the on-screen browser is unambiguously what they want — but
# the small router reflexively keeps webSearch and the model picks the
# invisible web path. Drop webSearch for that turn so controlBrowser
# wins. Only fires when a site is named AND we're in screen-share mode.
_site_tokens = (
"naver", "네이버", "google", "구글", "daum", "다음",
"youtube", "유튜브", "유투브", "bing",
)
if "controlBrowser" in routed_tools and "webSearch" in routed_tools \
and any(_tok in redacted.lower() for _tok in _site_tokens):
routed_tools = [t for t in routed_tools if t != "webSearch"]
debug_log("screen-share: site named — dropping webSearch so controlBrowser wins", "tools")
_planner_schema = generate_tools_json_schema(routed_tools, mcp_tools) _planner_schema = generate_tools_json_schema(routed_tools, mcp_tools)
_planner_tool_catalog: list[tuple[str, str]] = [] _planner_tool_catalog: list[tuple[str, str]] = []
for _schema in (_planner_schema or []): for _schema in (_planner_schema or []):
@@ -1095,6 +1295,15 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
needs_memory = False needs_memory = False
except Exception as exc: # noqa: BLE001 except Exception as exc: # noqa: BLE001
debug_log(f"recall gate failed (fail-open): {exc}", "memory") debug_log(f"recall gate failed (fail-open): {exc}", "memory")
# Conversational fast-path: when the router routed NO external-data tool,
# this is a greeting / small-talk / behavioural-instruction turn. Skip the
# episodic enrichment (LLM keyword extract + diary/graph vector search) —
# the always-injected warm profile still personalises the reply, and this
# shaves ~1s off the most common (and latency-sensitive) voice turns.
if needs_memory and not plan_demands_memory and not _router_wants_data:
debug_log("fast-path: no data tool routed — skipping episodic enrichment", "memory")
needs_memory = False
# Topic hint from the directive (if any) — passed to the memory # Topic hint from the directive (if any) — passed to the memory
# extractor so keyword selection is anchored on what the planner # extractor so keyword selection is anchored on what the planner
# actually wanted to look up, instead of re-deriving from the raw # actually wanted to look up, instead of re-deriving from the raw
@@ -1447,7 +1656,18 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# can't recognise. The markdown-fence format is explicit in the system prompt, so the # can't recognise. The markdown-fence format is explicit in the system prompt, so the
# model has a concrete template to follow. Using text tools from the start also avoids # model has a concrete template to follow. Using text tools from the start also avoids
# the wasted round-trip and prompt confusion of starting native and falling back mid-turn. # the wasted round-trip and prompt confusion of starting native and falling back mid-turn.
use_text_tools = (model_size == ModelSize.SMALL) # …BUT some small models emit clean native tool calls (qwen2.5/qwen3,
# llama3.x, mistral). Forcing text tools on those suppresses tool use almost
# entirely — the model just narrates ("부산 날씨는 맑습니다") and never emits a
# call, so getWeather/webSearch/controlBrowser never run. Use native for the
# tool-capable families (native still auto-falls-back to text on HTTP 400);
# only genuinely non-tool small models (e.g. gemma) default to text.
_model_l = (cfg.ollama_chat_model or "").lower()
_native_capable = any(k in _model_l for k in (
"qwen2.5", "qwen2", "qwen3", "llama3.1", "llama3.2", "llama3.3",
"mistral", "hermes", "command-r", "firefunction",
))
use_text_tools = (model_size == ModelSize.SMALL) and not _native_capable
prompts = get_system_prompts(model_size) prompts = get_system_prompts(model_size)
debug_log(f"Model size detected: {model_size.value} for {cfg.ollama_chat_model} (use_text_tools={use_text_tools})", "planning") debug_log(f"Model size detected: {model_size.value} for {cfg.ollama_chat_model} (use_text_tools={use_text_tools})", "planning")
@@ -1476,7 +1696,12 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
action_plan = strip_memory_directives(action_plan) action_plan = strip_memory_directives(action_plan)
_assistant_name = str(getattr(cfg, "wake_word", "jarvis") or "jarvis").strip().capitalize() _assistant_name = str(getattr(cfg, "wake_word", "jarvis") or "jarvis").strip().capitalize()
_persona_prompt = build_system_prompt(_assistant_name, os.environ.get("OUTPUT_LANGUAGE")) # Resolve once so the persona and the reply-language directive agree: the
# settings-UI value wins over the compose OUTPUT_LANGUAGE env (see
# _resolve_output_language). Building the persona from the raw env var while
# the directive used the config value made the two contradict each other.
_output_language = _resolve_output_language()
_persona_prompt = build_system_prompt(_assistant_name, _output_language)
def _build_initial_system_message() -> str: def _build_initial_system_message() -> str:
guidance = [_persona_prompt.strip()] guidance = [_persona_prompt.strip()]
@@ -1491,8 +1716,11 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# Placed at the FRONT (after the persona header) so a small model gives # Placed at the FRONT (after the persona header) so a small model gives
# it primacy over the persona's "use the user's language" lines — a tail # it primacy over the persona's "use the user's language" lines — a tail
# instruction loses to those when the query itself is in another language. # instruction loses to those when the query itself is in another language.
# Settings-UI value (config) wins over the compose OUTPUT_LANGUAGE env so
# changing the language in the settings page actually takes effect. Same
# resolved value feeds the persona above, so they can't diverge.
_lang_directive = reply_language_directive( _lang_directive = reply_language_directive(
os.environ.get("OUTPUT_LANGUAGE"), _output_language,
getattr(cfg, "tts_engine", "piper"), getattr(cfg, "tts_engine", "piper"),
) )
if _lang_directive: if _lang_directive:
@@ -1577,6 +1805,17 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# else: tools are passed via the native tools API parameter — do not include tools_desc # else: tools are passed via the native tools API parameter — do not include tools_desc
# here as well, since that confuses the model and causes it to not use tools properly. # here as well, since that confuses the model and causes it to not use tools properly.
# User-defined extra LLM instructions from the settings UI.
_user_instructions = str(_extra_config("llm_instructions", "")).strip()
if _user_instructions:
guidance.append("Additional instructions from the operator:\n" + _user_instructions)
# Recency reinforcement: repeat the language lock at the very END too.
# In a ~5k-token prompt the front-placed rule gets "lost in the middle";
# bigger models (qwen2.5:7b) otherwise leak Chinese/Cyrillic mid-reply.
if _lang_directive:
guidance.append(_lang_directive)
return "\n".join(guidance) return "\n".join(guidance)
messages = [] # type: ignore[var-annotated] messages = [] # type: ignore[var-annotated]

View File

@@ -36,6 +36,16 @@ _SYSTEM_PROMPT_TEMPLATE: str = (
"butler clichés, and never address the user as 'sir', 'madam', 'my liege', or similar. " "butler clichés, and never address the user as 'sir', 'madam', 'my liege', or similar. "
"Never stack multiple jokes in one reply. " "Never stack multiple jokes in one reply. "
"Be concise, conversational, and actionable. " "Be concise, conversational, and actionable. "
"This is a spoken voice assistant: answer in ONE short sentence whenever possible "
"(two at the very most). No lists, no preamble, no 'is there anything else' offers. "
"When a controlBrowser tool is available, use IT (never webSearch) for anything that "
"should happen in the on-screen browser — opening a site, searching on a site "
"(controlBrowser action 'search' with the right site), clicking, typing — because only "
"controlBrowser is visible on the broadcast; webSearch shows nothing on screen. "
"Never claim you performed an action — opening a website, navigating the browser, "
"playing or showing something on screen, changing a setting, sending a message — unless a "
"tool actually did it this turn and reported success. If you have no tool for what was asked, "
"or the tool failed, say so plainly; do not narrate a success that did not happen. "
"Never answer with a bare greeting like 'Hey there!', 'Hi!', 'Hello, how can I help you?', " "Never answer with a bare greeting like 'Hey there!', 'Hi!', 'Hello, how can I help you?', "
"'I hope you have a relaxing time today', or 'I'm here and ready to chat'. Always engage " "'I hope you have a relaxing time today', or 'I'm here and ready to chat'. Always engage "
"with the user's actual prompt, and when the 'Information the user has shared…' section is " "with the user's actual prompt, and when the 'Information the user has shared…' section is "
@@ -123,8 +133,12 @@ def output_language_directive(language: Optional[str]) -> Optional[str]:
f"CRITICAL OUTPUT RULE: write your ENTIRE reply only in {lang}. Even if " f"CRITICAL OUTPUT RULE: write your ENTIRE reply only in {lang}. Even if "
f"the user writes in English or any other language, you must still reply " f"the user writes in English or any other language, you must still reply "
f"only in {lang}. This rule overrides every other instruction about " f"only in {lang}. This rule overrides every other instruction about "
f"matching or using the user's language. Never mix in words, characters, " f"matching or using the user's language. Do NOT output a single Chinese/"
f"or punctuation from any other language or script." f"Hanja character, Japanese kana, Cyrillic letter, Arabic letter, or any "
f"other non-{lang} script anywhere in the reply — not even one word or "
f"clause. If a {lang} word exists, use it; never substitute or append a "
f"foreign-language equivalent. (Numerals and unavoidable proper-noun "
f"brand names are fine.)"
) )

View File

@@ -0,0 +1,169 @@
"""Operate the on-screen (Go-Live) Chrome like a person would.
Only meaningful in screen-share mode (``STREAM_BROWSER`` true): it drives the
on-screen Chrome via the Node CDP helper (``chrome-control.mjs``) so every
action is visible on the broadcast. Pointer moves, clicks and typing are real
xdotool input (visible cursor, char-by-char typing), never synthetic DOM
events.
This is the general browser-control tool (navigate to any site, manage
windows/tabs, click, type, scroll, screenshot). ``browseAndPlay`` remains the
YouTube-playback shortcut.
"""
from __future__ import annotations
import json
import os
import subprocess
from pathlib import Path
from typing import Dict, Any, Optional
from ..base import Tool, ToolContext
from ..types import ToolExecutionResult
from ...debug import debug_log
# .../owner/src/jarvis/tools/builtin/control_browser.py -> parents[4] == .../owner
_REPO_ROOT = Path(__file__).resolve().parents[4]
_NODE_SCRIPT = _REPO_ROOT / "bot" / "scripts" / "stream-test" / "chrome-control.mjs"
# Actions that don't change anything (safe, fast, no human-input latency).
_READ_ACTIONS = {"status", "listTabs"}
class ControlBrowserTool(Tool):
"""Drive the on-screen Chrome: navigate, manage tabs, click, type, scroll."""
@property
def name(self) -> str:
return "controlBrowser"
@property
def description(self) -> str:
return (
"Drive the on-screen Chrome that is shown on the broadcast, like a person would. "
"Use this (NOT webSearch) whenever the user wants something done or shown IN the "
"browser on screen: open a website or URL, search on a specific site (action "
"'search' with site=naver/google/daum/youtube/bing), go back/forward, refresh, "
"manage tabs (list/new/close/switch), close popups, click, type, scroll, or "
"screenshot. webSearch only returns text and shows nothing on screen; this tool "
"actually navigates the visible browser. Only available in screen-share mode. "
"Never claim you did any of this unless this tool returns success."
)
@property
def inputSchema(self) -> Dict[str, Any]:
return {
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": [
"status", "listTabs", "navigate", "search", "back",
"forward", "refresh", "newTab", "closeTab", "activateTab",
"closePopups", "click", "type", "scroll", "pressKey",
"screenshot",
],
"description": "What to do in the browser.",
},
"url": {"type": "string", "description": "Target URL/site for navigate/newTab (e.g. 'naver.com')."},
"query": {"type": "string", "description": "Search text for action 'search'."},
"site": {"type": "string", "description": "Search site for action 'search': naver, google, daum, youtube, bing."},
"index": {"type": "integer", "description": "Tab index for closeTab/activateTab (from listTabs)."},
"selector": {"type": "string", "description": "CSS selector for click/type."},
"text": {"type": "string", "description": "Text to type."},
"key": {"type": "string", "description": "Key to press, e.g. 'Return', 'Escape'."},
"dir": {"type": "string", "description": "Scroll direction: 'down' or 'up'."},
},
"required": ["action"],
}
def run(self, args: Optional[Dict[str, Any]], context: ToolContext) -> ToolExecutionResult:
cfg = context.cfg
if not getattr(cfg, "stream_browser", True):
return ToolExecutionResult(
success=False,
reply_text="화면 공유 모드(STREAM_BROWSER=true)에서만 브라우저를 제어할 수 있습니다.",
)
if not _NODE_SCRIPT.exists():
return ToolExecutionResult(success=False, reply_text="브라우저 제어 도구를 찾을 수 없습니다.")
if not args or not isinstance(args, dict):
return ToolExecutionResult(success=False, reply_text="수행할 동작(action)을 알려주세요.")
action = str(args.get("action", "")).strip()
if not action:
return ToolExecutionResult(success=False, reply_text="수행할 동작(action)을 알려주세요.")
# Pass the whole arg dict through as the command (the Node side reads the
# fields it needs per action).
command = json.dumps(args, ensure_ascii=False)
context.user_print(f"🖱️ 브라우저 제어: {action}")
debug_log(f" 🖱️ controlBrowser {command[:120]}", "tools")
# Human-input actions need time for the visible cursor move + char typing.
timeout = 25 if action in _READ_ACTIONS else 90
# Split deployment: when the browser (Chrome + X + xdotool) lives on a
# different machine, send the command to its control-server so the REAL
# input lands on that host's screen. Otherwise run chrome-control.mjs
# locally (all-in-one / browser-host layout).
control_url = os.environ.get("BROWSER_CONTROL_URL", "").strip()
try:
if control_url:
import urllib.request
req = urllib.request.Request(
control_url.rstrip("/") + "/control",
data=command.encode("utf-8"),
headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads((resp.read().decode("utf-8") or "").strip() or "{}")
else:
proc = subprocess.run(
["node", str(_NODE_SCRIPT), command],
capture_output=True,
text=True,
timeout=timeout,
env={**os.environ, "CDP_PORT": os.environ.get("CDP_PORT", "9222")},
)
data = json.loads((proc.stdout or "").strip() or "{}")
except Exception as e:
return ToolExecutionResult(success=False, reply_text=f"브라우저 제어에 실패했습니다: {e}")
if not data.get("ok"):
return ToolExecutionResult(
success=False, reply_text=f"브라우저 제어에 실패했습니다: {data.get('error', 'unknown')}"
)
# Return a factual summary of what ACTUALLY happened so the reply engine
# describes the real outcome and doesn't confabulate.
summary = self._summarise(action, args, data)
if action == "status":
# The broadcast (Go-Live) state lives in the bot runtime, surfaced
# to the tool via the turn context — report it alongside the tabs.
bc = getattr(context, "broadcasting", None)
state = "켜짐" if bc else ("꺼짐" if bc is not None else "알 수 없음")
summary = f"📡 방송: {state}\n{summary}"
return ToolExecutionResult(success=True, reply_text=summary)
@staticmethod
def _summarise(action: str, args: Dict[str, Any], data: Dict[str, Any]) -> str:
if action == "navigate":
return f"브라우저에서 {data.get('url', args.get('url'))} 로 이동했습니다."
if action == "search":
return f"{data.get('site', '')}에서 '{data.get('query', args.get('query'))}'를 검색해 화면에 띄웠습니다."
if action in ("back", "forward", "refresh"):
return f"브라우저: {action} 완료 ({data.get('url', '')})."
if action in ("status", "listTabs"):
tabs = data.get("tabs", [])
lines = "\n".join(f" [{t['index']}] {t.get('title') or t['url']}" for t in tabs) or " (탭 없음)"
return f"브라우저 탭 {len(tabs)}개:\n{lines}"
if action == "newTab":
return f"새 탭을 열었습니다 (index {data.get('index')})."
if action == "closeTab":
return f"{data.get('closed')}번을 닫았습니다 (남은 탭 {data.get('remaining')}개)."
if action == "activateTab":
return f"{data.get('active')}번으로 전환했습니다."
if action == "closePopups":
return f"팝업/빈 탭 {data.get('closed')}개를 닫았습니다."
if action == "screenshot":
return f"화면을 캡처했습니다: {data.get('path')}"
return "완료했습니다."

View File

@@ -175,6 +175,20 @@ WMO_CODES = {
99: "Thunderstorm with heavy hail", 99: "Thunderstorm with heavy hail",
} }
# Korean conditions for the concise spoken reply.
WMO_CODES_KO = {
0: "맑음", 1: "대체로 맑음", 2: "구름 조금", 3: "흐림",
45: "안개", 48: "서리 안개",
51: "약한 이슬비", 53: "이슬비", 55: "강한 이슬비",
56: "약한 어는 이슬비", 57: "강한 어는 이슬비",
61: "약한 비", 63: "", 65: "강한 비",
66: "약한 어는 비", 67: "강한 어는 비",
71: "약한 눈", 73: "", 75: "강한 눈", 77: "싸락눈",
80: "약한 소나기", 81: "소나기", 82: "강한 소나기",
85: "약한 눈소나기", 86: "강한 눈소나기",
95: "천둥번개", 96: "우박 동반 천둥번개", 99: "강한 우박 천둥번개",
}
class WeatherTool(Tool): class WeatherTool(Tool):
"""Tool for getting current weather using Open-Meteo API.""" """Tool for getting current weather using Open-Meteo API."""
@@ -412,71 +426,25 @@ class WeatherTool(Tool):
# Get weather description # Get weather description
weather_desc = WMO_CODES.get(weather_code, "Unknown conditions") weather_desc = WMO_CODES.get(weather_code, "Unknown conditions")
# Build response text — current conditions # Concise, ready-to-speak Korean one-liner for the voice path. The
lines = [ # tool result is normally re-synthesised by the LLM, but a small
f"Current weather in {location_display}:", # model rambles and leaks °F / CJK fragments, so we hand it a clean
f"", # Korean sentence it can echo verbatim (one-sentence system rule).
f"Conditions: {weather_desc}", _ko = WMO_CODES_KO.get(weather_code, weather_desc)
] _short_loc = location_display.split(",")[0].strip() or location_display
_ko_parts = [f"지금 {_short_loc} 날씨는 {_ko}"]
if temp_c is not None: if temp_c is not None:
lines.append(f"Temperature: {temp_c}°C ({temp_f}°F)") _t = f"기온 {round(temp_c)}"
if feels_like_c is not None and round(feels_like_c) != round(temp_c):
_t += f"(체감 {round(feels_like_c)}도)"
_ko_parts.append(_t)
ko_sentence = ", ".join(_ko_parts) + "입니다."
if feels_like_c is not None and feels_like_c != temp_c: # The reply is the clean Korean sentence ONLY — no English/°C source
lines.append(f"Feels like: {feels_like_c}°C ({feels_like_f}°F)") # for the model to echo ("25도 Celsius"), no forecast firehose to
# ramble over. The deterministic weather path in the engine returns
if humidity is not None: # this verbatim; on the LLM path the model just echoes one sentence.
lines.append(f"Humidity: {humidity}%") lines = [ko_sentence]
if wind_speed is not None:
wind_info = f"Wind: {wind_speed} km/h"
if wind_gusts and wind_gusts > wind_speed:
wind_info += f" (gusts up to {wind_gusts} km/h)"
lines.append(wind_info)
# Append today's hourly forecast (remaining hours)
hourly = weather_data.get("hourly", {})
hourly_times = hourly.get("time", [])
hourly_temps = hourly.get("temperature_2m", [])
hourly_codes = hourly.get("weather_code", [])
if hourly_times and hourly_temps:
# Get current hour from the current time field
current_time = current.get("time", "")
current_hour_str = current_time[11:13] if len(current_time) >= 13 else ""
current_hour = int(current_hour_str) if current_hour_str.isdigit() else 0
today_prefix = current_time[:10] if len(current_time) >= 10 else ""
hourly_lines = []
for i, t in enumerate(hourly_times):
if not t.startswith(today_prefix):
continue
hour_str = t[11:13] if len(t) >= 13 else ""
hour = int(hour_str) if hour_str.isdigit() else -1
# Show every 3 hours from now onwards
if hour > current_hour and hour % 3 == 0 and i < len(hourly_temps) and i < len(hourly_codes):
desc = WMO_CODES.get(hourly_codes[i], "")
hourly_lines.append(f" {hour:02d}:00 — {hourly_temps[i]}°C, {desc}")
if hourly_lines:
lines.append("")
lines.append("Today's forecast (upcoming hours):")
lines.extend(hourly_lines)
# Append daily forecast
daily = weather_data.get("daily", {})
daily_dates = daily.get("time", [])
daily_codes = daily.get("weather_code", [])
daily_max = daily.get("temperature_2m_max", [])
daily_min = daily.get("temperature_2m_min", [])
if daily_dates and daily_max and daily_min:
lines.append("")
lines.append("7-day forecast:")
for i, date_str in enumerate(daily_dates):
if i < len(daily_max) and i < len(daily_min) and i < len(daily_codes):
desc = WMO_CODES.get(daily_codes[i], "")
lines.append(f" {date_str}: {daily_min[i]}{daily_max[i]}°C, {desc}")
reply_text = "\n".join(lines) reply_text = "\n".join(lines)

View File

@@ -21,6 +21,7 @@ from .builtin.weather import WeatherTool
from .builtin.stop import StopTool from .builtin.stop import StopTool
from .builtin.tool_search import ToolSearchTool from .builtin.tool_search import ToolSearchTool
from .builtin.browse_and_play import BrowseAndPlayTool from .builtin.browse_and_play import BrowseAndPlayTool
from .builtin.control_browser import ControlBrowserTool
from .builtin.set_broadcast import SetBroadcastTool from .builtin.set_broadcast import SetBroadcastTool
from .types import ToolExecutionResult from .types import ToolExecutionResult
from ..config import Settings from ..config import Settings
@@ -42,6 +43,7 @@ BUILTIN_TOOLS = {
"stop": StopTool(), "stop": StopTool(),
"toolSearchTool": ToolSearchTool(), "toolSearchTool": ToolSearchTool(),
"browseAndPlay": BrowseAndPlayTool(), "browseAndPlay": BrowseAndPlayTool(),
"controlBrowser": ControlBrowserTool(),
"setBroadcast": SetBroadcastTool(), "setBroadcast": SetBroadcastTool(),
} }

View File

@@ -0,0 +1,74 @@
"""The locked reply language must have a single source of truth.
Regression: the persona prompt was built from the raw ``OUTPUT_LANGUAGE`` env
while the reply-language directive read the settings-UI value (config JSON).
Changing the language in the settings page rewrote the directive but left the
persona contradicting it. ``_resolve_output_language`` is now the one resolver
both call sites use, so they cannot diverge.
"""
import pytest
@pytest.mark.unit
def test_settings_value_wins_over_env(monkeypatch, tmp_path):
from jarvis.reply.engine import _resolve_output_language
cfg_path = tmp_path / "config.json"
cfg_path.write_text('{"output_language": "Korean"}', encoding="utf-8")
monkeypatch.setenv("JARVIS_CONFIG_PATH", str(cfg_path))
monkeypatch.setenv("OUTPUT_LANGUAGE", "English")
# The settings page value must take effect over the compose env default.
assert _resolve_output_language() == "Korean"
@pytest.mark.unit
def test_env_used_when_settings_absent(monkeypatch, tmp_path):
from jarvis.reply.engine import _resolve_output_language
cfg_path = tmp_path / "config.json"
cfg_path.write_text("{}", encoding="utf-8")
monkeypatch.setenv("JARVIS_CONFIG_PATH", str(cfg_path))
monkeypatch.setenv("OUTPUT_LANGUAGE", "English")
assert _resolve_output_language() == "English"
@pytest.mark.unit
def test_unset_when_neither_configured(monkeypatch, tmp_path):
from jarvis.reply.engine import _resolve_output_language
cfg_path = tmp_path / "config.json"
cfg_path.write_text("{}", encoding="utf-8")
monkeypatch.setenv("JARVIS_CONFIG_PATH", str(cfg_path))
monkeypatch.delenv("OUTPUT_LANGUAGE", raising=False)
# Empty string or None both mean "no lock" downstream; normalise the check.
assert not _resolve_output_language()
@pytest.mark.unit
def test_persona_and_directive_agree_on_settings_value(monkeypatch, tmp_path):
"""End-to-end: the same resolved value feeds the persona and the directive,
so a settings-UI language can't be honoured by one and ignored by the other.
"""
from jarvis.reply.engine import _resolve_output_language
from jarvis.system_prompt import build_system_prompt, reply_language_directive
cfg_path = tmp_path / "config.json"
cfg_path.write_text('{"output_language": "Korean"}', encoding="utf-8")
monkeypatch.setenv("JARVIS_CONFIG_PATH", str(cfg_path))
monkeypatch.setenv("OUTPUT_LANGUAGE", "English")
lang = _resolve_output_language()
persona = build_system_prompt("Jarvis", lang)
directive = reply_language_directive(lang, "melo")
# Persona's user-language clause is rewritten to Korean, not English...
assert "in Korean" in persona
assert "in English" not in persona
# ...and the directive locks to the same Korean. (The directive may name
# English as a counter-example - "even if the user writes in English" - so
# we assert the lock target, not the mere absence of the word "English".)
assert directive is not None and "Korean" in directive

View File

@@ -0,0 +1,119 @@
"""End-to-end persistence of the output_language settings change.
Closes the loop the reviewer flagged: a language chosen in the settings web UI
must (1) take effect immediately for the reply engine and (2) survive a
container recreate. The pieces:
bridge._save() -> writes BOTH /data/jarvis-settings.json (persistent)
and JARVIS_CONFIG_PATH (live runtime config)
entrypoint merge -> on recreate, re-renders config from the env template
then merges the persistent overrides back on top
engine._resolve_output_language() -> reads JARVIS_CONFIG_PATH, config wins
over the OUTPUT_LANGUAGE env
This test drives the REAL bridge save function and the REAL engine resolver
(the resolver is loaded standalone because the full engine import needs the
mcp package, which isn't installed in CI here). It simulates the env default
disagreeing with the chosen language, which is exactly the bug condition.
"""
import ast
import json
import os
from pathlib import Path
import pytest
# bridge.settings_web imports only stdlib at module load (flask is imported
# lazily inside register()), so it is safe to import directly.
import sys
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "bridge"))
import settings_web # noqa: E402
def _load_resolver():
"""Load engine._resolve_output_language + _extra_config without importing
the heavy jarvis package (which pulls in the optional mcp dependency)."""
src = (
Path(__file__).resolve().parents[1]
/ "src/jarvis/reply/engine.py"
).read_text("utf-8")
tree = ast.parse(src)
wanted = {"_extra_config", "_resolve_output_language"}
mod = ast.Module(
body=[
n
for n in tree.body
if isinstance(n, ast.FunctionDef) and n.name in wanted
],
type_ignores=[],
)
ns = {"os": os, "Optional": __import__("typing").Optional}
exec(compile(mod, "engine_subset", "exec"), ns) # noqa: S102
return ns["_resolve_output_language"]
def _simulate_recreate_merge(template_lang: str, config_path: Path, persist_path: Path):
"""Mirror docker/entrypoint.sh: re-render the runtime config from the env
template, then merge the persistent overrides on top."""
config_path.write_text(json.dumps({"output_language": template_lang}), "utf-8")
if persist_path.exists():
base = json.loads(config_path.read_text("utf-8"))
ov = json.loads(persist_path.read_text("utf-8"))
base.update(ov)
config_path.write_text(json.dumps(base, ensure_ascii=False, indent=2), "utf-8")
@pytest.mark.integration
def test_settings_save_applies_and_survives_recreate(monkeypatch, tmp_path):
config_path = tmp_path / "jarvis.json"
persist_path = tmp_path / "data" / "jarvis-settings.json"
# The compose env default is the "old" language that must be overridden.
monkeypatch.setenv("OUTPUT_LANGUAGE", "English")
monkeypatch.setenv("JARVIS_CONFIG_PATH", str(config_path))
monkeypatch.setenv("JARVIS_SETTINGS_PATH", str(persist_path))
# Start from the env-rendered config (as entrypoint would produce).
config_path.write_text(json.dumps({"output_language": "English"}), "utf-8")
resolve = _load_resolver()
# Before the change: the env default wins.
assert resolve() == "English"
# 1) User saves Korean in the settings UI.
settings_web._save({"output_language": "Korean"})
# Both targets are written.
assert json.loads(config_path.read_text("utf-8"))["output_language"] == "Korean"
assert json.loads(persist_path.read_text("utf-8"))["output_language"] == "Korean"
# 2) Applies immediately: the resolver now returns Korean (config > env).
assert resolve() == "Korean"
# 3) Survives a container recreate: entrypoint re-renders the config from the
# env template (still English) then merges the persistent override.
_simulate_recreate_merge("English", config_path, persist_path)
assert json.loads(config_path.read_text("utf-8"))["output_language"] == "Korean"
assert resolve() == "Korean"
@pytest.mark.integration
def test_persona_and_directive_follow_persisted_language(monkeypatch, tmp_path):
"""After persistence, the persona and the reply directive both lock to the
saved language, not the env default."""
from jarvis.system_prompt import build_system_prompt, reply_language_directive
config_path = tmp_path / "jarvis.json"
persist_path = tmp_path / "data" / "jarvis-settings.json"
monkeypatch.setenv("OUTPUT_LANGUAGE", "English")
monkeypatch.setenv("JARVIS_CONFIG_PATH", str(config_path))
monkeypatch.setenv("JARVIS_SETTINGS_PATH", str(persist_path))
config_path.write_text(json.dumps({"output_language": "English"}), "utf-8")
settings_web._save({"output_language": "Korean"})
lang = _load_resolver()()
persona = build_system_prompt("Jarvis", lang)
directive = reply_language_directive(lang, "melo")
assert "in Korean" in persona and "in English" not in persona
assert directive is not None and "Korean" in directive