14 Commits

Author SHA1 Message Date
javis-bot
ffc16665e5 fix(controlBrowser): never report moveMouse/search success without a real move
Some checks failed
Release / semantic-release (push) Successful in 19s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m15s
Release / build-linux (push) Failing after 7m7s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
moveMouse returned ok:true even when humanHover did nothing (no on-screen box)
or the selector never matched — recreating the "claims it moved but didn't"
bug. Now: humanHover returns a boolean (and brings the element into view first);
moveMouse returns ok:false when the target isn't found or has no on-screen box,
and when site=naver/... whose box isn't on the current page it navigates to the
site home first before hovering. search now reports input=human|api-fallback|api
so a silent fallback to cursor-less DOM input is visible, and the tool surfaces
that note in the reply instead of implying a human-like search happened.
2026-06-24 19:17:46 +09:00
javis-bot
5629da7e9f feat(controlBrowser): add moveMouse (hover) action for the visible cursor
The tool had no cursor-move/hover action, so "move the mouse to the search box"
had nothing to call and a weak model just claimed it had moved it. Add a
moveMouse action wired to human.humanHover (real xdotool cursor), targetable by
CSS selector or site= (that site's search box). Also clarify in the tool
description that 'navigate' types into the address bar (no mouse) while
'search'/'type' move the real cursor to the on-page box and click before
typing, so the model picks the visible-cursor path when the user wants it.
2026-06-24 19:14:04 +09:00
javis-bot
83999a5b0b fix(prompts): classify every sub-8B model (2b/4b/5b/6b) as SMALL
Some checks failed
Release / semantic-release (push) Successful in 24s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m21s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
detect_model_size only matched 1b/3b/7b, so a genuinely small model such as
qwen*:4b fell through to LARGE and got the terse, less-guided prompt set it
can't follow — contributing to off-tone, rambling replies. Extend the small
patterns to cover all sub-8B sizes (1b-7b) across :/-/_ separators and sync the
spec table.
2026-06-24 18:31:54 +09:00
javis-bot
d5fd218c86 fix(reply): stop weak models parroting persona example facts
A 4b model replied to "하이" with "테니스 연습을 Trenches Gym에서..." — it copied
the literal "box at Trenches Gym" few-shot example embedded in the persona
prompt and mangled boxing into tennis, presenting a prompt example as if it
were a real user fact. Remove the copyable proper-noun example and add an
explicit guard: use ONLY names/places/activities that literally appear in the
memory section, never borrow them from the instructions or example wording.
2026-06-24 18:31:54 +09:00
javis-bot
23b1fe692b docs: warn against setting OLLAMA_INTENT_MODEL larger than the chat model
Some checks failed
Release / semantic-release (push) Successful in 20s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m19s
Release / build-linux (push) Failing after 7m13s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
A deployment had OLLAMA_INTENT_MODEL=qwen2.5:7b while the chat model was a 4b,
so every auxiliary call (intent judge, tool router, place extraction, query
decomposition) ran on the bigger, slower model and added latency to each
command. Make the .env.example comment state the invariant explicitly.
2026-06-24 17:57:30 +09:00
javis-bot
7bb9718c34 feat(reply): cap spoken replies at a single sentence
Replies stayed long because the prompt stack gave conflicting length signals:
the persona said "one sentence (two at the very most)" AND told the model to
"state the answer in a sentence, then add a dry observation" (a 2nd sentence),
while voice_style said "two to three sentences maximum". The model followed the
longest. Make all three sources agree on exactly one sentence: the persona's
aside must now fold into the same sentence as a trailing clause (never a 2nd
sentence), voice_style caps at one sentence, and agents/llm.md says 한 문장.
Shorter replies also cut Edge-TTS latency, since synth time scales with text
length. Specs (prompts.spec.md) and docs/llm_contexts.md updated; deterministic
prompt-contract tests added.
2026-06-24 17:55:27 +09:00
javis-bot
7da2fcb5e5 feat(stt): beam-search decoding + no prev-text conditioning for accuracy
Whisper was decoding with beam_size=1 (greedy), the least accurate setting,
which hurt recognition on short/accented/noisy Discord-mic speech. Switch the
default to beam search (5, Whisper's own default) and stop conditioning on the
previous clip's transcript (which causes repetition/drift on isolated short
utterances rather than helping). Both are env-tunable (STT_BEAM_SIZE,
STT_CONDITION_ON_PREV) so accuracy/latency can be traded without a code change;
wired into docker-compose and documented in .env.example.
2026-06-24 17:55:20 +09:00
javis-bot
680f5a656a docs: reflect the separate auxiliary intent/router model in llm_contexts + README
Some checks failed
Release / semantic-release (push) Successful in 18s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m18s
Release / build-linux (push) Failing after 7m6s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
Follow-up to the OLLAMA_INTENT_MODEL split: document that the Docker stack runs
intent judging / tool routing / extraction on a small qwen2.5:3b (pulled by
ollama-init) kept separate from the big chat answer model, and that setting
OLLAMA_INTENT_MODEL = OLLAMA_CHAT_MODEL folds them back onto one resident model.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 17:38:58 +09:00
javis-bot
b52ffd2b18 perf: run auxiliary LLM calls on a small model, big model only for the answer
Some checks failed
Release / semantic-release (push) Successful in 19s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / build-linux (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
tests / Unit tests (Linux, Python 3.11) (push) Has been cancelled
Intent judging, tool routing and arg extraction are classification/JSON calls,
not the spoken answer, yet the stack aliased OLLAMA_INTENT_MODEL back to the big
chat model — so each command paid the big model's cost 2-3 times for routing
before the reply even ran. With the GPU on, that round-trip stacking is the main
remaining per-turn latency. Default OLLAMA_INTENT_MODEL to qwen2.5:3b (the
project's reference small model, clean Korean on classification) and have
ollama-init pull it. The reply engine already routes these calls through
intent_judge_model, so answer quality is untouched; set OLLAMA_INTENT_MODEL =
OLLAMA_CHAT_MODEL to fold back onto one resident model.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 17:35:40 +09:00
javis-bot
140fc56f18 feat: play the Nth YouTube result in browseAndPlay via an index arg
Some checks failed
Release / semantic-release (push) Successful in 22s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m10s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
agents/llm.md promises "play the Nth video from the top", but browseAndPlay
only ever clicked the first result. Add an optional 1-based index argument
(default 1, backward-compatible) threaded to the Node helper, which now clicks
the Nth a#video-title and clamps to the number of results returned so asking
beyond the list plays the last available video instead of failing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 15:33:45 +09:00
javis-bot
5ee47827f3 perf: cap chat output tokens via ollama_num_predict to bound reply latency
Spoken (TTS) replies are 1-2 sentences, so an unbounded num_predict only
exposes the worst case where the chat model rambles or loops. Add an
ollama_num_predict config (default 512, 0 disables) wired into the reply
loop's chat call on both the native- and text-tool paths. The 512-token
headroom stays well above this app's short tool-call JSON, so capping never
truncates a tool call. This keeps the user's quality model instead of
downgrading it. Configurable in the container via OLLAMA_NUM_PREDICT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 15:33:45 +09:00
javis-bot
c189ce2e65 feat: Korean Chrome locale, agents/llm.md voice instructions, drop emoji rule
Some checks failed
Release / semantic-release (push) Successful in 26s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m17s
Release / build-linux (push) Failing after 7m7s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
- run-chrome.sh: render web content (YouTube/Google/Naver) in Korean. --lang
  only sets Chrome's UI; the Accept-Language sent to sites comes from the
  profile's intl.accept_languages, which a persisted user-data-dir kept at
  en-US. Seed the profile to ko-KR and add --accept-lang=ko-KR,ko.
- agents/llm.md: runtime instructions for the reply LLM (loaded by the agents
  feature) — name "자비스", concise 1-2 sentence TTS replies (no emojis/markdown),
  nuance-based intent, and YouTube voice controls (open/search/play Nth/pause/
  back) via the on-screen browser tool.
- CLAUDE.md: drop the "use emojis in CLI output" rule — this assistant replies by
  Discord voice, not CLI, so output should be plain.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 04:34:17 +09:00
javis-bot
086dd5cde7 fix: accept edge as a valid tts_engine and migrate stale persisted engines
Some checks failed
Release / semantic-release (push) Successful in 34s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m17s
Release / build-linux (push) Failing after 7m9s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
load_settings() coerced any tts_engine outside {piper, chatterbox} to piper, so
with TTS_ENGINE=edge the reply engine saw "piper" and treated the voice as
English-only in reply_language_directive() (only the OUTPUT_LANGUAGE lock kept
replies Korean). Add "edge" (and "melo") to the accepted set so the engine is
labelled multilingual correctly.

Also: a stale tts_engine in the persistent /data/jarvis-settings.json (melo/xtts
from an earlier voice, no longer built) would override the configured engine via
the entrypoint merge and leave the bot silent. Reset those to the env engine
during the merge.

Verified: load_settings() with tts_engine=edge now returns "edge"; the merge
maps melo/xtts -> edge; reply_language_directive("edge") is multilingual; 27
tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 03:49:53 +09:00
javis-bot
f64d76e737 feat: use Edge TTS (Korean Hyunsu voice @ +45%) as the default voice
Some checks failed
Release / semantic-release (push) Successful in 31s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / build-linux (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
tests / Unit tests (Linux, Python 3.11) (push) Has been cancelled
The user chose Microsoft Edge TTS, voice ko-KR-HyunsuMultilingualNeural at rate
+45% (~1.45x), as the natural Korean voice. Wire it into the bridge and make it
the default engine.

- bridge/server.py: _edge_synthesize() calls edge-tts and transcodes the MP3 to
  PCM16 mono WAV with the system ffmpeg (temp file for a correct header);
  TTS_ENGINE default -> edge; EDGE_TTS_VOICE / EDGE_TTS_RATE env-driven
- requirements-bridge.txt: add edge-tts (lightweight; httpx)
- compose/.env.example/README: TTS_ENGINE=edge + EDGE_TTS_* knobs; note the
  online/privacy trade-off (reply text is sent to Microsoft, needs internet)
- drop the now-unused MeloTTS build layer (Dockerfile) and melo-worker
  (supervisord) — edge synthesises in-process, no model/worker baked, slimmer
  and faster image; settings UI engine list -> edge/piper, restart only bridge

Verified on host: edge-tts -> ffmpeg yields a valid 16-bit mono 24kHz WAV;
envsubst renders tts_engine=edge; docker build --check + 26 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 03:44:15 +09:00
33 changed files with 782 additions and 175 deletions

View File

@@ -34,18 +34,18 @@ WHISPER_DEVICE=cuda
WHISPER_COMPUTE_TYPE=float16
# Optional explicit Piper voice model (.onnx). If empty, the jarvis default is used.
TTS_PIPER_MODEL_PATH=
# TTS engine: "melo" (default) uses the MeloTTS Korean voice served by the warm
# melo-worker (Korean speaker, speed 1.5). Set to "piper" to use Piper directly.
TTS_ENGINE=melo
# Melo-only by default: if MeloTTS synthesis fails the bridge returns no audio
# rather than speaking Korean through the English Piper voice (which mangles it).
# Set to 1 only if you explicitly want the Piper fallback.
# TTS engine: "edge" (default) uses Microsoft Edge TTS, a natural Korean neural
# voice. Set to "piper" for the offline English voice. NOTE: edge is ONLINE —
# reply text is sent to Microsoft's servers and needs internet.
TTS_ENGINE=edge
# Edge voice + speaking rate. Rate is a percentage (+45% ≈ 1.45×). Korean voices:
# ko-KR-HyunsuMultilingualNeural (M), ko-KR-InJoonNeural (M), ko-KR-SunHiNeural (F).
EDGE_TTS_VOICE=ko-KR-HyunsuMultilingualNeural
EDGE_TTS_RATE=+45%
# Neural-only by default: if synthesis fails the bridge returns no audio rather
# than speaking Korean through the English Piper voice. Set to 1 to allow the
# Piper fallback.
MELO_FALLBACK_PIPER=0
# Where the bridge reaches the in-container MeloTTS worker, and how long it
# waits for a synthesis. Speaking rate is set on the worker via MELO_SPEED.
MELO_WORKER_URL=http://127.0.0.1:8770
MELO_TIMEOUT=30
MELO_SPEED=1.5
# ---------------------------------------------------------------------------
# Jarvis brain (Ollama-backed). In Docker these populate the rendered
@@ -59,11 +59,15 @@ OLLAMA_BASE_URL=http://127.0.0.1:11434
# free-form chit-chat. Swap back to qwen3:8b for the strongest tool-calling.
OLLAMA_CHAT_MODEL=qwen2.5:3b
# Model for the auxiliary small-model calls: intent judge, tool router, weather
# place extraction, query decomposition. BLANK (default) reuses OLLAMA_CHAT_MODEL
# so the stack runs on one already-warm model. The code's built-in default
# (gemma4:e2b) is NOT pulled by this stack, so leaving this unset previously made
# every router/extractor call silently fail. Only set this if you also pull the
# model into Ollama.
# place extraction, query decomposition. These are classification/JSON calls,
# NOT the spoken answer, so a small fast model here cuts 2-3 big-model round
# trips per command without touching answer quality. BLANK uses the stack
# default qwen2.5:3b, which ollama-init pulls automatically. Set it equal to
# OLLAMA_CHAT_MODEL to run everything on one resident model instead (saves VRAM
# at the cost of slower routing when the chat model is large).
# NEVER set this LARGER than OLLAMA_CHAT_MODEL: the auxiliary calls would then
# run on the bigger, slower model and add latency to every command (the exact
# opposite of the split's purpose). Keep it <= the chat model, blank, or equal.
OLLAMA_INTENT_MODEL=
OLLAMA_EMBED_MODEL=nomic-embed-text
WHISPER_MODEL=small
@@ -226,6 +230,7 @@ COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
# OLLAMA_CHAT_MODEL=qwen2.5:7b # quality (needs ~5GB VRAM + whisper small)
# OLLAMA_CHAT_MODEL=qwen2.5:3b # speed (fits easily, faster on 8GB GPUs)
# WHISPER_MODEL=small # small frees VRAM for a bigger LLM; medium=more accurate
# STT_BEAM_SIZE=5 # beam search (5) > greedy (1) for accuracy; lower for speed
# MELO_DEVICE=cuda # cpu if no GPU on the bot host
# --- Settings web UI (http://localhost:8765/settings on the bot host) ---

View File

@@ -1,6 +1,6 @@
Data privacy comes first, always.
All user-facing command line output should make use of emojis. Especially an initial emoji to start off the lines that depict what the line is about. Output should make use of indentation spacing to establish a visual hierarchy and aim to make output as easy to sift through as possible. Exception: Windows .bat scripts cannot use emojis (cmd.exe doesn't render Unicode properly).
This assistant is used through a Discord bot with voice (TTS) replies, not a CLI. Do not add emojis to user-facing assistant output. Keep output plain and readable. (Runtime assistant behaviour lives in `agents/*.md`, which is injected into the reply LLM's prompt.)
Any important point in our logical flows should have debug logs using the `debug_log` method from `src/jarvis/debug.py`. Avoid excessive logging to keep the logs easily readable and actionable.

View File

@@ -65,20 +65,14 @@ RUN ls -d /opt/venv/lib/python*/site-packages/nvidia/cublas/lib \
> /etc/ld.so.conf.d/nvidia-cu12.conf 2>/dev/null \
&& /sbin/ldconfig || true
# --- MeloTTS Korean voice (separate /opt/melo py3.11 venv; see setup-melo.sh).
# Heavy layer (torch CPU + transformers + MeCab); placed before the app
# COPY so it stays cached across source-only changes. ---
COPY docker/setup-melo.sh /app/docker/setup-melo.sh
# Strip CR before running: a Windows checkout (autocrlf) yields CRLF, which makes
# bash read line 18 as `set -euxo pipefail\r` and abort with
# "set: pipefail: invalid option name". .gitattributes pins *.sh to LF, but this
# keeps the build working even on a not-yet-renormalised working tree.
RUN sed -i 's/\r$//' /app/docker/setup-melo.sh && bash /app/docker/setup-melo.sh
# --- Korean voice: Microsoft Edge TTS (online neural). No model is baked — the
# `edge-tts` pip package (in requirements-bridge.txt) calls the MS service at
# runtime and the bridge transcodes the MP3 to PCM16 with ffmpeg. No heavy
# TTS build layer is needed. ---
# --- Human input + window management for the on-screen Chrome control tool.
# Placed AFTER the heavy melo layer so it doesn't bust that cache. xdotool
# injects real X pointer/keyboard events (visible cursor, char-by-char
# typing) into the broadcast; wmctrl lists/moves windows. ---
# xdotool injects real X pointer/keyboard events (visible cursor,
# char-by-char typing) into the broadcast; wmctrl lists/moves windows. ---
RUN apt-get update && apt-get install -y --no-install-recommends \
xdotool wmctrl \
&& rm -rf /var/lib/apt/lists/*

View File

@@ -69,7 +69,7 @@ docker compose -f docker-compose.yml -f docker-compose.gpu-linux.yml up -d --bui
docker compose -f docker-compose.yml -f docker-compose.gpu-windows.yml up -d --build
# ── GPU 없이 (CPU 전용 호스트) ──
# .env 에 WHISPER_DEVICE=cpu, MELO_DEVICE=cpu 를 넣고 베이스만 사용
# .env 에 WHISPER_DEVICE=cpu 를 넣고 베이스만 사용
docker compose up -d --build
```
@@ -87,7 +87,7 @@ COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml
> Linux와 Windows는 GPU를 컨테이너에 넣는 방식이 달라서 override 파일이 갈립니다. Linux는 CDI(`devices: nvidia.com/gpu=all`), Windows(Docker Desktop)는 Compose의 `deploy.resources.reservations.devices`(`driver: nvidia`)를 씁니다. 호스트 사전 준비는 아래 "GPU 가속" 절 참고.
`docker compose up` 한 번이면 자동으로:
- Ollama 서버가 뜨고, `ollama-init`이 채팅/임베딩 모델을 **자동 pull**
- Ollama 서버가 뜨고, `ollama-init`이 채팅/보조(의도·라우팅)/임베딩 모델을 **자동 pull** (보조 모델 `OLLAMA_INTENT_MODEL`은 기본 `qwen2.5:3b`로, 큰 채팅 모델은 답변에만 쓰고 내부 분류 호출은 이 작은 모델이 처리)
- VNC+XFCE 데스크톱 + 크롬 + Python 브릿지가 기동
- Whisper STT 모델 / Piper TTS 음성 자동 다운로드(볼륨에 캐시)
@@ -113,7 +113,7 @@ docker compose up -d # 유저봇이 로그인해 지정 음성채널에
### GPU 가속 (OS별)
LLM(Ollama), Whisper STT, MeloTTS가 GPU에서 돕니다(env 기본 `WHISPER_DEVICE=cuda`, `MELO_DEVICE=cuda`). NVIDIA Blackwell(sm_120, 예: RTX 5050/5070Ti)에서 검증: 컨테이너 내 torch cu128 CUDA 동작, Ollama GPU 오프로드, faster-whisper float16, MeloTTS GPU 합성 모두 확인.
LLM(Ollama) Whisper STT가 GPU에서 돕니다(env 기본 `WHISPER_DEVICE=cuda`). TTS는 기본값이 Edge TTS(온라인 한국어 음성)라 GPU를 쓰지 않습니다. NVIDIA Blackwell(sm_120, 예: RTX 5050/5070Ti)에서 검증: 컨테이너 내 torch cu128 CUDA 동작, Ollama GPU 오프로드, faster-whisper float16 모두 확인.
GPU는 위 "실행 — Docker"의 OS별 override 파일로 켜집니다. 호스트 사전 준비는 OS마다 다릅니다:
@@ -137,7 +137,7 @@ docker run --rm --device nvidia.com/gpu=all ubuntu nvidia-smi -L # GPU 보이
**공통:**
- 모델: 베이스 compose 기본은 `qwen2.5:3b`(8GB VRAM에서 도구호출 안정적). 더 무겁게(`qwen2.5:7b`, `qwen3:8b` 등) 쓰려면 `.env``OLLAMA_CHAT_MODEL` 변경.
- **GPU가 없거나 인식 실패 시 자동으로 CPU 폴백**(Whisper)하므로 안전합니다. 명시적으로 CPU만 쓰려면 override 파일 없이 베이스만 올리고 `.env``WHISPER_DEVICE=cpu`, `MELO_DEVICE=cpu`를 두세요.
- **GPU가 없거나 인식 실패 시 자동으로 CPU 폴백**(Whisper)하므로 안전합니다. 명시적으로 CPU만 쓰려면 override 파일 없이 베이스만 올리고 `.env``WHISPER_DEVICE=cpu`를 두세요.
- 데이터(메모리 DB), Whisper 캐시, Piper 음성은 named volume에 영속됩니다.
- 셀프봇 영상 송출 의존성은 이미지에 기본 포함하지 않습니다. 쓰려면 컨테이너에서 `cd /app/bot && bun add discord.js-selfbot-v13 @dank074/discord-video-stream` 후 재시작(또는 Dockerfile에 추가).
@@ -243,7 +243,7 @@ cd bot; bun run register; bun run start # 창 2: (일반 봇이면) 슬래시
- `BRIDGE_URL` — 봇이 호출할 브릿지 주소 (기본 `http://127.0.0.1:8765`)
- `STREAM_BACKEND`, `DISCORD_SELFBOT_TOKEN`, `NOVNC_URL` — 화면 송출
- `VNC_DISPLAY=:1`, `VNC_RESOLUTION`, `VNC_FRAMERATE`, `VNC_BITRATE_KBPS` — 캡처
- `WHISPER_DEVICE/COMPUTE_TYPE`, `MELO_DEVICE` — GPU 호스트면 `cuda`/`float16`, CPU 전용이면 `cpu`(GPU 자체는 OS별 override compose 파일로 켬)
- `WHISPER_DEVICE/COMPUTE_TYPE` — GPU 호스트면 `cuda`/`float16`, CPU 전용이면 `cpu`(GPU 자체는 OS별 override compose 파일로 켬)
- `OLLAMA_CHAT_MODEL` — 두뇌 LLM (기본 `qwen2.5:3b`)
- `COMPOSE_FILE` — OS별 GPU override를 매번 `-f`로 안 치고 자동 적용 (위 "실행 — Docker" 참고)
- `output_language` — 출력 언어 고정(비우면 사용자 언어). 설정 웹 UI(`/settings`)에서 바꾸면 env 기본값보다 우선하며 컨테이너 재생성 후에도 유지됩니다.

13
agents/llm.md Normal file
View File

@@ -0,0 +1,13 @@
# 자비스 운영자 지시
- 너의 이름은 자비스다.
- 모든 답변은 음성(TTS)으로 읽혀 나간다. 그러니 무조건 한 문장으로만 답한다. 두 문장 이상 쓰지 않는다. 목록, 마크다운, 이모지, 그리고 소리 내어 읽기 어려운 특수문자는 쓰지 않는다.
- 정해진 문구에만 반응하지 말고, 실제 사람처럼 말의 뉘앙스와 맥락으로 의도를 알아듣고 처리한다.
화면 속 크롬(방송 화면)에서 유튜브를 다룰 때 (화면에 보여야 하므로 반드시 on-screen 브라우저 제어 도구로 수행한다):
- "유튜브 켜줘" → 방송 크롬에서 유튜브를 연다.
- "유튜브에서 OO 검색해줘" → 유튜브로 가서 검색창에 OO를 사람이 직접 타이핑하듯 입력하고 검색한다.
- "위에서 N번째 영상 재생해줘" 또는 "왼쪽에서 N번째 영상 재생해줘" → 검색 결과 목록에서 그 위치의 영상을 재생한다.
- "일시정지해줘" → 현재 영상을 일시정지한다. "다시 재생해줘" → 이어서 재생한다.
- "영상 종료" 또는 "그만 보여줘" → 뒤로 가서 직전 화면으로 돌아간다.

View File

@@ -2,10 +2,11 @@
// 9222) so the action is visible on the Go-Live broadcast, and prints a JSON
// result on stdout for the Python `browseAndSearch` tool to wrap.
//
// node browse-search.mjs "<query>" [search|youtube]
// node browse-search.mjs "<query>" [search|youtube] [index]
//
// - search : Google-search the query, return the top organic results.
// - youtube : search YouTube and play the first result.
// - youtube : search YouTube and play a result. `index` is the 1-based position
// from the top of the result list (default 1 = first result).
//
// Backend selection for `search`:
// 1. The broadcast Chrome over CDP (visible on the Go-Live stream).
@@ -29,6 +30,9 @@ const UA =
'(KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36';
const query = process.argv[2] || '';
const mode = (process.argv[3] || 'search').toLowerCase();
// 1-based position of the YouTube result to play, counted from the top of the
// list. Defaults to 1 (first result). Anything <1 or non-numeric falls back to 1.
const playIndex = Math.max(1, parseInt(process.argv[4], 10) || 1);
const out = (o) => { process.stdout.write(JSON.stringify(o)); };
if (!query) { out({ ok: false, error: 'no query' }); process.exit(1); }
@@ -105,15 +109,21 @@ try {
await page.bringToFront().catch(() => {});
if (mode === 'youtube') {
// Type into YouTube's search box like a person, then play the first result.
// Type into YouTube's search box like a person, then play the requested
// result (the Nth from the top of the list; default the first).
await typeSearch('https://www.youtube.com/?hl=ko', 'input#search, input[name="search_query"]', query);
await page.waitForSelector('ytd-video-renderer a#video-title, a#video-title', { timeout: 20000 });
const first = page.locator('ytd-video-renderer a#video-title, a#video-title').first();
const title = (await first.getAttribute('title').catch(() => '')) || (await first.innerText().catch(() => ''));
await first.click();
const results = page.locator('ytd-video-renderer a#video-title, a#video-title');
// Clamp to what's actually on the page so "play the 5th" still plays the
// last available result rather than failing when fewer were returned.
const available = await results.count();
const targetIdx = Math.min(playIndex, Math.max(available, 1)) - 1;
const target = results.nth(targetIdx);
const title = (await target.getAttribute('title').catch(() => '')) || (await target.innerText().catch(() => ''));
await target.click();
await page.waitForSelector('#movie_player', { timeout: 20000 });
await page.evaluate(() => { const v = document.querySelector('video'); if (v && v.paused) v.play(); });
out({ ok: true, mode, title: (title || '').trim(), url: page.url() });
out({ ok: true, mode, index: targetIdx + 1, title: (title || '').trim(), url: page.url() });
} else {
// Type into Google's search box like a person, then read the results.
await typeSearch('https://www.google.com/?hl=ko', 'textarea[name="q"], input[name="q"]', query);

View File

@@ -17,6 +17,7 @@
// status | listTabs
// navigate {url} | back | forward | refresh
// newTab {url?} | closeTab {index?} | activateTab {index} | closePopups
// moveMouse {selector | site} (hover the real cursor, no click)
// click {selector} | type {text, selector?} | scroll {dir, notches?}
// pressKey {key} | screenshot {path}
import { chromium } from 'playwright';
@@ -40,6 +41,15 @@ if (!action) { out({ ok: false, error: 'no action' }); process.exit(1); }
const norm = (u) => (/^https?:\/\//i.test(u) ? u : `https://${u}`);
// Per-site homepage + search-box selector, shared by `search` and `moveMouse`.
const SITES = {
naver: { home: 'https://www.naver.com', box: '#query, input[name="query"]' },
google: { home: 'https://www.google.com', box: 'textarea[name="q"], input[name="q"]' },
daum: { home: 'https://www.daum.net', box: '#q, input[name="q"]' },
youtube: { home: 'https://www.youtube.com', box: 'input#search, input[name="search_query"]' },
bing: { home: 'https://www.bing.com', box: '#sb_form_q, input[name="q"]' },
};
// The genuinely-active tab is the one whose document is visible. Playwright has
// no "active page" accessor over CDP, so probe visibilityState (fixes treating
// tab 0 as active and breaking sequential ops on a specific tab).
@@ -103,13 +113,6 @@ try {
const q = String(cmd.query || '').trim();
if (!q) throw new Error('search: no query');
const siteKey = String(cmd.site || 'google').toLowerCase();
const SITES = {
naver: { home: 'https://www.naver.com', box: '#query, input[name="query"]' },
google: { home: 'https://www.google.com', box: 'textarea[name="q"], input[name="q"]' },
daum: { home: 'https://www.daum.net', box: '#q, input[name="q"]' },
youtube: { home: 'https://www.youtube.com', box: 'input#search, input[name="search_query"]' },
bing: { home: 'https://www.bing.com', box: '#sb_form_q, input[name="q"]' },
};
const s = SITES[siteKey] || SITES.google;
await front(page);
// 1) Go to the homepage.
@@ -122,23 +125,31 @@ try {
// 2) Click the on-page search box, type the query, submit.
const box = page.locator(s.box).first();
await box.waitFor({ state: 'visible', timeout: 15000 }).catch(() => {});
// Report which input path actually ran: 'human' = real xdotool cursor
// move + char typing; 'api-fallback' = the humanClick path threw and we
// fell back to cursor-less DOM click/fill; 'api' = no xdotool at all. This
// makes "did the cursor really move" verifiable from the result.
let searchInput;
if (HAS_XDOTOOL && cmd.human !== false) {
try {
await human.humanClick(page, box);
await human.humanType(q);
await human.pressKey('Return');
searchInput = 'human';
} catch {
searchInput = 'api-fallback';
await box.click().catch(() => {});
await box.fill(q).catch(() => {});
await page.keyboard.press('Enter').catch(() => {});
}
} else {
searchInput = 'api';
await box.click().catch(() => {});
await box.fill(q);
await page.keyboard.press('Enter');
}
await page.waitForLoadState('domcontentloaded').catch(() => {});
out({ ok: true, site: SITES[siteKey] ? siteKey : 'google', query: q, url: page.url(), title: await page.title().catch(() => '') });
out({ ok: true, site: SITES[siteKey] ? siteKey : 'google', query: q, url: page.url(), title: await page.title().catch(() => ''), input: searchInput });
break;
}
@@ -205,6 +216,45 @@ try {
break;
}
case 'moveMouse': {
// Move/hover the REAL cursor onto an element WITHOUT clicking. Target is a
// CSS selector, or site=naver/google/... for that site's search box.
// Only meaningful with xdotool (the visible cursor); with no xdotool there
// is no cursor to move, so report that rather than faking success. Every
// failure to actually move (no xdotool, selector never matches, element
// has no on-screen box) returns ok:false — we must never claim the cursor
// moved when it did not (the exact bug the user reported).
const siteKey = String(cmd.site || '').toLowerCase();
const selector = String(cmd.selector || '').trim() || (SITES[siteKey] ? SITES[siteKey].box : '');
if (!selector) throw new Error('moveMouse: no selector or known site');
if (!(HAS_XDOTOOL && cmd.human !== false)) {
out({ ok: false, error: 'no xdotool: cannot move the visible cursor on this host' });
break;
}
await front(page);
let locator = page.locator(selector).first();
let visible = await locator.waitFor({ state: 'visible', timeout: 8000 }).then(() => true).catch(() => false);
// A named site whose search box isn't on the current page: go to its home
// first (real omnibox), then target the box there.
if (!visible && SITES[siteKey]) {
try { await human.navigateOmnibox(SITES[siteKey].home); await page.waitForLoadState('domcontentloaded').catch(() => {}); }
catch { await page.goto(SITES[siteKey].home, { waitUntil: 'domcontentloaded' }).catch(() => {}); }
locator = page.locator(SITES[siteKey].box).first();
visible = await locator.waitFor({ state: 'visible', timeout: 8000 }).then(() => true).catch(() => false);
}
if (!visible) {
out({ ok: false, error: `moveMouse: target not found (${cmd.selector || siteKey})` });
break;
}
const moved = await human.humanHover(page, locator);
if (!moved) {
out({ ok: false, error: 'moveMouse: element has no on-screen box; cursor not moved' });
break;
}
out({ ok: true, target: cmd.selector || siteKey, input: 'human' });
break;
}
case 'click': {
const selector = String(cmd.selector || '').trim();
if (!selector) throw new Error('click: no selector');

View File

@@ -136,14 +136,18 @@ export async function navigateOmnibox(text) {
}
// Move the real cursor over an element (hover, no click) - e.g. to reveal a
// video player's controls or to focus it for a keyboard shortcut.
// video player's controls or to focus it for a keyboard shortcut. Returns true
// only if the element had an on-screen box and the cursor was actually moved;
// returns false when there is nothing to move to (so callers must not report
// success). Brings the element into view with a real wheel scroll first.
export async function humanHover(page, locator) {
const box = await locator.boundingBox().catch(() => null);
if (!box) return;
const box = await bringIntoView(page, locator);
if (!box) return false;
const g = await page.evaluate(() => ({ sx: window.screenX, sy: window.screenY, ow: window.outerWidth, oh: window.outerHeight, iw: window.innerWidth, ih: window.innerHeight }));
const bx = Math.max(0, Math.round((g.ow - g.iw) / 2));
const oy = g.sy + Math.max(0, g.oh - g.ih - bx);
await humanMove(Math.round(g.sx + bx + box.x + box.width * 0.5), Math.round(oy + box.y + box.height * 0.4));
return true;
}
export { sleep, rand };

View File

@@ -21,7 +21,11 @@ nvidia-cudnn-cu12
# --- Bridge HTTP service ---
flask>=3.0.0
# --- Text-to-speech (Piper) ---
# --- Text-to-speech ---
# Edge TTS: the primary Korean voice (online MS neural). Lightweight (httpx);
# emits MP3, transcoded to PCM16 by the system ffmpeg in the bridge.
edge-tts>=6.1.0
# Piper: offline English fallback.
piper-tts>=1.3.0
# --- Built-in tools (lazily imported; needed for full functionality) ---

View File

@@ -87,12 +87,22 @@ VAD_MIN_SPEECH_MS = int(os.environ.get("VAD_MIN_SPEECH_MS", "200"))
# Korean phrase decoded as Chinese) and shaves a little latency. Empty = auto.
STT_LANGUAGE = os.environ.get("STT_LANGUAGE", "ko").strip() or None
# TTS engine: "melo" (MeloTTS Korean speaker, the warm worker) is the primary
# voice; Piper is kept as a fallback if the worker is unreachable. Set
# TTS_ENGINE=piper to disable MeloTTS entirely.
# Whisper decoding accuracy knobs. beam_size=1 is greedy decoding — fast but the
# least accurate; beam search (5 is the Whisper default) explores alternatives
# and noticeably improves recognition on short, accented, or noisy Discord-mic
# speech. condition_on_previous_text=False stops Whisper from feeding a previous
# clip's transcript back in as a prompt, which on isolated short utterances
# causes repetition loops and drift rather than helping. Both are env-tunable so
# accuracy/latency can be traded without a code change (lower STT_BEAM_SIZE for
# speed, raise it for accuracy).
STT_BEAM_SIZE = max(1, int(os.environ.get("STT_BEAM_SIZE", "5")))
STT_CONDITION_ON_PREV = os.environ.get("STT_CONDITION_ON_PREV", "0") in ("1", "true", "True", "yes", "on")
# TTS engine: "edge" (Microsoft Edge TTS, natural Korean neural voice) is the
# primary voice. "melo" (a warm MeloTTS worker) and "piper" remain selectable.
def _tts_engine_setting() -> str:
"""TTS engine: settings-UI value (runtime config JSON) wins, else env, else
melo. Read at startup; the settings UI restarts the bridge on apply."""
edge. Read at startup; the settings UI restarts the bridge on apply."""
try:
_cp = os.environ.get("JARVIS_CONFIG_PATH", "/app/config/jarvis.json")
_v = json.loads(open(_cp, encoding="utf-8").read()).get("tts_engine")
@@ -100,16 +110,22 @@ def _tts_engine_setting() -> str:
return str(_v).strip().lower()
except Exception:
pass
return os.environ.get("TTS_ENGINE", "melo").strip().lower()
return os.environ.get("TTS_ENGINE", "edge").strip().lower()
TTS_ENGINE = _tts_engine_setting()
# Edge TTS (online MS neural voice). Voice + rate are env-driven so they can be
# changed without code. Default: Korean "Hyunsu" multilingual voice at +45%
# (≈1.45×), the chosen settings. NOTE: edge synthesis sends the reply TEXT to
# Microsoft's servers and needs internet — an intentional privacy trade-off for
# the more natural voice.
EDGE_TTS_VOICE = os.environ.get("EDGE_TTS_VOICE", "ko-KR-HyunsuMultilingualNeural").strip()
EDGE_TTS_RATE = os.environ.get("EDGE_TTS_RATE", "+45%").strip()
MELO_WORKER_URL = os.environ.get("MELO_WORKER_URL", "http://127.0.0.1:8770")
MELO_TIMEOUT = float(os.environ.get("MELO_TIMEOUT", "30"))
# When MeloTTS is the engine, do NOT silently fall back to the English Piper
# voice on failure: speaking Korean text through an English voice produces
# mangled audio. Default is melo-only (return no audio on failure); set
# MELO_FALLBACK_PIPER=1 to opt into the Piper fallback.
# Do NOT silently fall back to the English Piper voice on a neural-voice failure:
# speaking Korean through an English voice produces mangled audio. Default is
# neural-only (return no audio on failure); set MELO_FALLBACK_PIPER=1 to opt in.
MELO_FALLBACK_PIPER = os.environ.get("MELO_FALLBACK_PIPER", "0") in ("1", "true", "True", "yes", "on")
# ---------------------------------------------------------------------------
@@ -238,7 +254,12 @@ def transcribe(wav_bytes: bytes) -> dict:
print("[bridge] no speech detected (VAD) — skipping STT", flush=True)
return {"text": "", "language": None, "note": "음성 아님(VAD 차단)"}
segments, info = _whisper.transcribe(audio, beam_size=1, language=STT_LANGUAGE)
segments, info = _whisper.transcribe(
audio,
beam_size=STT_BEAM_SIZE,
language=STT_LANGUAGE,
condition_on_previous_text=STT_CONDITION_ON_PREV,
)
# Second line of defence: drop non-speech / hallucinated segments by
# Whisper's own no_speech_prob. The no_speech_prob hard cutoff (plus the VAD
# pre-gate above) is what rejects noise/hallucinations. The avg_logprob
@@ -302,6 +323,54 @@ def _coerce_bool(value) -> Optional[bool]:
return str(value).strip().lower() in ("1", "true", "yes", "on")
def _edge_synthesize(text: str) -> Optional[bytes]:
"""Synthesise via Microsoft Edge TTS (online neural voice) and return a
16-bit PCM WAV, or None on any failure. Edge emits MP3; we transcode to
PCM16 mono with the system ffmpeg, writing to a temp file (seekable) so the
WAV header carries a correct length. Needs internet."""
import asyncio
import subprocess
import tempfile
try:
import edge_tts # type: ignore
async def _gen() -> bytes:
comm = edge_tts.Communicate(text, EDGE_TTS_VOICE, rate=EDGE_TTS_RATE)
buf = bytearray()
async for chunk in comm.stream():
if chunk.get("type") == "audio":
buf.extend(chunk["data"])
return bytes(buf)
mp3 = asyncio.run(_gen())
if not mp3:
print("[bridge] edge TTS returned no audio", flush=True)
return None
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as t:
out_path = t.name
try:
proc = subprocess.run(
["ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-i", "pipe:0", "-ac", "1", "-ar", "24000",
"-acodec", "pcm_s16le", out_path],
input=mp3, capture_output=True,
)
if proc.returncode != 0:
print(f"[bridge] edge ffmpeg transcode failed: {proc.stderr.decode('utf-8','ignore')[:200]}", flush=True)
return None
with open(out_path, "rb") as f:
return f.read()
finally:
try:
os.unlink(out_path)
except OSError:
pass
except Exception as e: # pragma: no cover - network / dep dependent
print(f"[bridge] edge synth failed: {e}", flush=True)
return None
def _melo_synthesize(text: str) -> Optional[bytes]:
"""Synthesise via the warm MeloTTS worker (separate /opt/melo venv, Korean
speaker @ speed 1.5). Returns a 16-bit PCM WAV, or None on any failure so
@@ -361,20 +430,22 @@ def _tts_ready() -> bool:
def synthesize(text: str) -> Optional[bytes]:
"""Synthesize text to a 16-bit PCM WAV. The primary voice is MeloTTS
(Korean speaker, speed 1.5) served by the warm melo worker; Piper is a
fallback if the worker is unavailable. Returns None if TTS is off."""
"""Synthesize text to a 16-bit PCM WAV. The primary voice is Edge TTS (a
natural Korean neural voice); "melo" uses the warm MeloTTS worker. For a
neural engine, Piper (English) is only used if explicitly enabled, since
speaking Korean through an English voice mangles it. Returns None if off."""
if not TTS_ENABLED or not text.strip():
return None
if TTS_ENGINE == "melo":
audio = _melo_synthesize(text)
_neural = {"edge": _edge_synthesize, "melo": _melo_synthesize}.get(TTS_ENGINE)
if _neural is not None:
audio = _neural(text)
if audio:
return audio
if not MELO_FALLBACK_PIPER:
# Melo-only: better silent than mangled English for Korean text.
print("[bridge] melo synth failed; no audio (Piper fallback disabled)", flush=True)
# Neural-only: better silent than mangled English for Korean text.
print(f"[bridge] {TTS_ENGINE} synth failed; no audio (Piper fallback disabled)", flush=True)
return None
print("[bridge] melo synth failed; falling back to Piper", flush=True)
print(f"[bridge] {TTS_ENGINE} synth failed; falling back to Piper", flush=True)
return _piper_synthesize(text)

View File

@@ -22,8 +22,7 @@ from typing import Any, Dict
FIELDS = [
("ollama_chat_model", "LLM 모델", "model"),
("whisper_model", "STT(Whisper) 모델", "select:tiny,base,small,medium,large,large-v3"),
("tts_engine", "TTS 엔진", "select:melo,piper"),
("melo_speed", "TTS 속도 (MeloTTS)", "number:0.5:2.5:0.1"),
("tts_engine", "TTS 엔진", "select:edge,piper"),
("output_language", "출력 언어 (비우면 사용자 언어)", "text"),
("llm_thinking_enabled", "LLM 사고(thinking) 모드", "bool"),
("agentic_max_turns", "에이전트 최대 반복", "number:1:12:1"),
@@ -54,9 +53,7 @@ def _current() -> Dict[str, Any]:
cfg = _read_config()
out: Dict[str, Any] = {}
for k in _KEYS:
if k == "melo_speed":
out[k] = cfg.get("melo_speed", os.environ.get("MELO_SPEED", "1.5"))
elif k == "output_language":
if k == "output_language":
out[k] = cfg.get("output_language", os.environ.get("OUTPUT_LANGUAGE", ""))
else:
out[k] = cfg.get(k, "")
@@ -78,12 +75,7 @@ def _coerce(updates: Dict[str, Any]) -> Dict[str, Any]:
for k, v in updates.items():
if k not in _KEYS:
continue
if k == "melo_speed":
try:
v = float(v)
except (TypeError, ValueError):
continue
elif k == "agentic_max_turns":
if k == "agentic_max_turns":
try:
v = int(v)
except (TypeError, ValueError):
@@ -114,15 +106,15 @@ def _save(updates: Dict[str, Any]) -> None:
def _apply() -> str:
# Restart melo + bridge AFTER this response is sent. Detached (new session)
# so the bridge being killed mid-restart doesn't drop the restart itself,
# and the HTTP client still receives this response.
# Restart the bridge AFTER this response is sent. Detached (new session) so
# the bridge being killed mid-restart doesn't drop the restart itself, and
# the HTTP client still receives this response. (Edge TTS has no worker.)
try:
subprocess.Popen(
["sh", "-c", "sleep 1; supervisorctl restart melo-worker bridge"],
["sh", "-c", "sleep 1; supervisorctl restart bridge"],
start_new_session=True,
)
return "1초 후 브리지/TTS 워커가 재시작되어 반영됩니다."
return "1초 후 브리지가 재시작되어 반영됩니다."
except Exception as e: # pragma: no cover
return str(e)

View File

@@ -40,6 +40,9 @@ services:
environment:
OLLAMA_HOST: http://ollama:11434
CHAT_MODEL: ${OLLAMA_CHAT_MODEL:-qwen2.5:3b}
# Small auxiliary model for intent/router/extraction calls (see javis
# service). Pulled here so the split is ready out of the box.
INTENT_MODEL: ${OLLAMA_INTENT_MODEL:-qwen2.5:3b}
EMBED_MODEL: ${OLLAMA_EMBED_MODEL:-nomic-embed-text}
entrypoint: ["/bin/sh", "-c"]
command:
@@ -48,6 +51,10 @@ services:
until ollama list >/dev/null 2>&1; do sleep 2; done;
echo "[ollama-init] pulling $$CHAT_MODEL";
ollama pull "$$CHAT_MODEL";
if [ -n "$$INTENT_MODEL" ] && [ "$$INTENT_MODEL" != "$$CHAT_MODEL" ]; then
echo "[ollama-init] pulling $$INTENT_MODEL (auxiliary intent/router model)";
ollama pull "$$INTENT_MODEL";
fi;
echo "[ollama-init] pulling $$EMBED_MODEL";
ollama pull "$$EMBED_MODEL";
echo "[ollama-init] models ready.";
@@ -62,22 +69,27 @@ services:
# Point the brain at the ollama service and the bot at the in-container bridge.
OLLAMA_BASE_URL: http://ollama:11434
OLLAMA_CHAT_MODEL: ${OLLAMA_CHAT_MODEL:-qwen2.5:3b}
# Auxiliary small-model calls (intent judge, tool router, arg extraction,
# query decomposition) run on this fast model so the big chat model only
# runs for the actual spoken answer. With the GPU on, this is the main
# per-turn latency win: a command no longer pays the big model's cost 2-3
# times for routing/extraction. Defaults to qwen2.5:3b (the project's
# reference small model, clean Korean on classification); set it equal to
# OLLAMA_CHAT_MODEL to fold everything back onto one resident model.
OLLAMA_INTENT_MODEL: ${OLLAMA_INTENT_MODEL:-qwen2.5:3b}
OLLAMA_EMBED_MODEL: ${OLLAMA_EMBED_MODEL:-nomic-embed-text}
WHISPER_MODEL: ${WHISPER_MODEL:-medium}
WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
# TTS engine. Rendered into /app/config/jarvis.json via envsubst (the
# bridge reads that JSON BEFORE the env, so it must carry the real engine,
# not the template's old hardcoded "piper" — otherwise Korean text is read
# by the English Piper voice). Default melo; .env can override.
TTS_ENGINE: ${TTS_ENGINE:-melo}
# MeloTTS on the GPU (cu128 torch baked by docker/setup-melo.sh). CPU synth
# serialised under load and pushed TTS to 7-8s; GPU does ~0.3s/sentence.
MELO_DEVICE: ${MELO_DEVICE:-cuda}
# Speaking rate for MeloTTS. Set here (with a default) so supervisord's
# %(ENV_MELO_SPEED)s passthrough always resolves and an .env override
# actually reaches the melo-worker. Lower it (e.g. 1.1) for a calmer pace.
MELO_SPEED: ${MELO_SPEED:-1.5}
# not a hardcoded one — otherwise Korean text is read by the English Piper
# voice). Default edge; .env can override (e.g. piper for offline).
TTS_ENGINE: ${TTS_ENGINE:-edge}
# Edge TTS voice + rate (the chosen natural Korean voice). NOTE: edge is an
# ONLINE engine — reply text is sent to Microsoft and needs internet.
EDGE_TTS_VOICE: ${EDGE_TTS_VOICE:-ko-KR-HyunsuMultilingualNeural}
EDGE_TTS_RATE: ${EDGE_TTS_RATE:-+45%}
# Optional single-language lock for replies (empty = user's own language).
OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-ko}
# Drop the pre-loop planner LLM call to cut voice-reply latency on small
@@ -85,6 +97,9 @@ services:
PLANNER_ENABLED: ${PLANNER_ENABLED:-0}
# Lock STT to Korean (skip Whisper auto-detect).
STT_LANGUAGE: ${STT_LANGUAGE:-ko}
# Whisper decode accuracy: beam search (5) over greedy (1) lifts recognition
# on short/noisy Discord speech. Lower to 1 for minimum latency.
STT_BEAM_SIZE: ${STT_BEAM_SIZE:-5}
VOICE_SILENCE_MS: ${VOICE_SILENCE_MS:-600}
BRIDGE_URL: http://127.0.0.1:8765
# Split-deployment role: full (default, all-in-one), browser (only the

View File

@@ -10,12 +10,15 @@ set -euo pipefail
: "${OLLAMA_BASE_URL:=http://ollama:11434}"
: "${OLLAMA_CHAT_MODEL:=qwen3:8b}"
# Auxiliary small-model calls (intent judge, tool router, weather place
# extraction, query decomposition). The code default is gemma4:e2b, which this
# stack does not pull, so those calls would silently fail and fall open —
# crippling tool routing and arg extraction. Reuse the (already warm) chat model
# by default so everything runs on one resident model; override if you pull a
# dedicated small model.
: "${OLLAMA_INTENT_MODEL:=${OLLAMA_CHAT_MODEL}}"
# extraction, query decomposition). Default to a small fast model so the big
# chat model only runs for the actual spoken answer — the main per-turn latency
# win once the GPU is in use, since the 2-3 routing/extraction calls per command
# no longer pay the big model's cost. ollama-init pulls this model. Set it equal
# to OLLAMA_CHAT_MODEL to fold everything back onto one resident model.
: "${OLLAMA_INTENT_MODEL:=qwen2.5:3b}"
# Cap chat-model output tokens per turn (worst-case latency guard). Spoken
# answers are 1-2 sentences; 512 is safe headroom above tool-call JSON. 0 = off.
: "${OLLAMA_NUM_PREDICT:=512}"
: "${OLLAMA_EMBED_MODEL:=nomic-embed-text}"
: "${WHISPER_MODEL:=small}"
: "${WHISPER_DEVICE:=cuda}"
@@ -32,7 +35,7 @@ set -euo pipefail
: "${XDG_RUNTIME_DIR:=/run/user/0}"
: "${PULSE_SERVER:=unix:${XDG_RUNTIME_DIR}/pulse/native}"
export VNC_RESOLUTION OLLAMA_BASE_URL OLLAMA_CHAT_MODEL OLLAMA_INTENT_MODEL OLLAMA_EMBED_MODEL \
export VNC_RESOLUTION OLLAMA_BASE_URL OLLAMA_CHAT_MODEL OLLAMA_NUM_PREDICT OLLAMA_INTENT_MODEL OLLAMA_EMBED_MODEL \
WHISPER_MODEL WHISPER_DEVICE WHISPER_COMPUTE_TYPE JARVIS_DB_PATH \
PIPER_VOICE PIPER_VOICE_DIR TTS_PIPER_MODEL_PATH BRIDGE_HOST BRIDGE_PORT \
XDG_RUNTIME_DIR PULSE_SERVER
@@ -51,12 +54,18 @@ export JARVIS_CONFIG_PATH=/app/config/jarvis.json
# the env-rendered config, so changes survive container recreate.
if [ -f /data/jarvis-settings.json ]; then
python3 - <<'PY' || true
import json
import json, os
try:
base = json.load(open("/app/config/jarvis.json"))
ov = json.load(open("/data/jarvis-settings.json"))
if isinstance(base, dict) and isinstance(ov, dict):
base.update(ov)
# A stale persisted tts_engine from an earlier voice (melo/xtts, no
# longer built into the image) would override the configured engine and
# leave the bot silent. Reset those to the env-configured engine.
if base.get("tts_engine") in ("melo", "xtts"):
base["tts_engine"] = os.environ.get("TTS_ENGINE", "edge")
print(f"[entrypoint] reset stale tts_engine -> {base['tts_engine']}")
json.dump(base, open("/app/config/jarvis.json", "w"), ensure_ascii=False, indent=2)
print("[entrypoint] merged persistent settings overrides")
except Exception as e:

View File

@@ -4,6 +4,7 @@
"ollama_base_url": "${OLLAMA_BASE_URL}",
"ollama_embed_model": "${OLLAMA_EMBED_MODEL}",
"ollama_chat_model": "${OLLAMA_CHAT_MODEL}",
"ollama_num_predict": "${OLLAMA_NUM_PREDICT}",
"intent_judge_model": "${OLLAMA_INTENT_MODEL}",
"tts_enabled": true,
"tts_engine": "${TTS_ENGINE}",

View File

@@ -18,6 +18,27 @@ cat > /etc/opt/chrome/policies/managed/jarvis.json <<'JSON'
{ "CommandLineFlagSecurityWarningsEnabled": false }
JSON
# Seed the profile's web-content language to Korean so sites (YouTube, Google,
# Naver) render in Korean. --lang sets Chrome's own UI, but the Accept-Language
# sent to sites comes from the profile's intl.accept_languages, which a persisted
# user-data-dir would otherwise keep at en-US regardless of --accept-lang.
PREFS_DIR="${CHROME_PROFILE_DIR:-/root/chrome-profile}/Default"
PREFS="${PREFS_DIR}/Preferences"
mkdir -p "$PREFS_DIR"
if [ -f "$PREFS" ]; then
python3 - "$PREFS" <<'PY' 2>/dev/null || true
import json, sys
p = sys.argv[1]
d = json.load(open(p))
d.setdefault("intl", {})
d["intl"]["accept_languages"] = "ko-KR,ko"
d["intl"]["selected_languages"] = "ko-KR,ko"
json.dump(d, open(p, "w"), ensure_ascii=False)
PY
else
printf '%s' '{"intl":{"accept_languages":"ko-KR,ko","selected_languages":"ko-KR,ko"}}' > "$PREFS"
fi
# Minimal, non-automation flags. --remote-debugging exposes CDP so the brain can
# drive this on-screen Chrome (Google/YouTube/Naver), --disable-features=Translate
# hides the translate popup. NO --test-type / --disable-blink-features.
@@ -26,6 +47,7 @@ exec google-chrome \
--no-default-browser-check \
--disable-features=Translate,TranslateUI \
--lang=ko-KR \
--accept-lang=ko-KR,ko \
--remote-debugging-port="${CDP_PORT:-9222}" \
--remote-debugging-address="${CDP_BIND:-127.0.0.1}" \
--user-data-dir="${CHROME_PROFILE_DIR:-/root/chrome-profile}" \

View File

@@ -49,31 +49,8 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:melo-worker]
; Warm MeloTTS Korean voice (speed 1.5) in its own py3.11 venv. The bridge's
; synthesize() POSTs here; if this is down the bridge falls back to Piper.
command=/app/docker/run-if-role.sh full,bot /opt/melo/bin/python /app/bridge/melo_worker.py
directory=/app
; HF_HOME points at the dedicated, image-baked melo cache (warmed in
; setup-melo.sh). The brain's whisper_cache volume is mounted over
; /root/.cache/huggingface, so without this the pre-cached BERT + KR checkpoint
; would be shadowed and re-downloaded (and would fail if the host is offline).
; HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE force pure-cache reads: the pinned old
; transformers/huggingface_hub otherwise retry the network on every load and
; error out instead of falling back to the (complete) baked cache.
; MELO_DEVICE and MELO_SPEED inherit from the container env (compose sets both
; with defaults: cuda / 1.5) so the worker runs MeloTTS on the GPU at the
; configured rate. supervisord interpolates %(ENV_x)s from its own environment,
; which is the container's — so MELO_SPEED must always be set in the env
; (compose guarantees it) or this expansion fails at startup. Hardcoding 1.5
; here previously shadowed the .env value, so lowering MELO_SPEED had no effect.
environment=MELO_LANGUAGE="KR",MELO_SPEED="%(ENV_MELO_SPEED)s",MELO_DEVICE="%(ENV_MELO_DEVICE)s",MELO_WORKER_HOST="127.0.0.1",MELO_WORKER_PORT="8770",HF_HOME="/opt/melo-cache",HF_HUB_OFFLINE="1",TRANSFORMERS_OFFLINE="1"
priority=280
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
# (No TTS worker program: the default Edge TTS engine synthesises in-process in
# the bridge via the `edge-tts` package — no warm model/worker is needed.)
[program:bridge]
command=/app/docker/run-if-role.sh full,bot /opt/venv/bin/python -m bridge.server

View File

@@ -19,14 +19,14 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
- Time + location context (re-injected each turn)
- Tool schema: native via `generate_tools_json_schema()` ([src/jarvis/tools/registry.py](src/jarvis/tools/registry.py)) or text fallback via `_text_tool_call_guidance()` ([engine.py:68](src/jarvis/reply/engine.py:68))
- Tool results from prior turns (raw or digested — see #5)
- **Output**: OpenAI-style `{content, tool_calls, thinking}`. Consumed by the tool orchestrator and TTS pipeline. Natural-language content is delivered immediately; no post-turn evaluator runs.
- **Limits**: `num_ctx: 8192` (explicit). Timeout `llm_chat_timeout_sec` (45s). Auto-fallback from native to text tool-calls on HTTP 400 (`ToolsNotSupportedError`), sticky for the session. Risk: `fetch_web_page` truncates at 50,000 chars (~37k tokens) — mitigated for SMALL models by tool-result digest (#5) which compresses the payload before it enters the messages history. LARGE models receive the raw payload and may silently see a truncated context.
- **Output**: OpenAI-style `{content, tool_calls, thinking}`. Consumed by the tool orchestrator and TTS pipeline. Natural-language content is delivered immediately; no post-turn evaluator runs. Spoken-answer length: the persona (`system_prompt.py`) and `voice_style` (`prompts/system.py`) both constrain the reply to a SINGLE sentence — any dry aside must fold into that one sentence as a trailing clause, never a second sentence. This keeps TTS latency down (synth time scales with text length) and matches the `agents/llm.md` operator instruction.
- **Limits**: `num_ctx: 8192` (explicit). Output `num_predict: cfg.ollama_num_predict` (default 512, `0`/negative disables) caps generated tokens per turn — a worst-case latency guard for short spoken answers; the headroom stays above tool-call JSON so it does not truncate tool calls (both native and text tool-call paths). Timeout `llm_chat_timeout_sec` (45s). Auto-fallback from native to text tool-calls on HTTP 400 (`ToolsNotSupportedError`), sticky for the session. Risk: `fetch_web_page` truncates at 50,000 chars (~37k tokens) — mitigated for SMALL models by tool-result digest (#5) which compresses the payload before it enters the messages history. LARGE models receive the raw payload and may silently see a truncated context.
## 2. Intent Judge
- **File**: [src/jarvis/listening/intent_judge.py](src/jarvis/listening/intent_judge.py) — `IntentJudge.evaluate()`.
- **Trigger**: on a speech segment *only if* there is an engagement signal (wake word detected, hot-window active, or TTS playing). Pure ambient speech skips it.
- **Model / gating**: `cfg.intent_judge_model` (default `gemma4:e2b`, ~2B). Falls back to text-based wake detection if Ollama is unavailable.
- **Model / gating**: `cfg.intent_judge_model`. Code-level default `gemma4:e2b` (~2B); the **Docker stack** renders it from `OLLAMA_INTENT_MODEL` (default `qwen2.5:3b`, pulled by `ollama-init`), kept deliberately **separate from `ollama_chat_model`** so this judge and the tool router (#3, #7) run on a small fast model while the big chat model is reserved for the spoken answer. Setting `OLLAMA_INTENT_MODEL = OLLAMA_CHAT_MODEL` folds them back onto one resident model. Falls back to text-based wake detection if Ollama is unavailable.
- **Inputs**:
- Rolling transcript buffer (last 120s, with timestamps)
- Wake-word timestamp (if any), normalised aliases
@@ -246,7 +246,7 @@ user input
3. Pre-warm the intent-judge model before TTS finishes.
4. Cache tool-router (#7) output by query hash.
5. Give each digest its own timeout budget rather than sharing `llm_digest_timeout_sec` (today a slow memory digest can starve the max-turn digest).
6. Consider single-model deployments: router+planner prefer `intent_judge_model`; loading a second model hurts cold-start latency on small hardware.
6. Two-model vs single-model tradeoff: the Docker default keeps a **separate** small `intent_judge_model` (`OLLAMA_INTENT_MODEL=qwen2.5:3b`) so routing/judging/extraction don't pay the big chat model's per-call cost — the main win once the GPU holds both models resident. On VRAM-constrained hardware, fold them onto one model by setting `OLLAMA_INTENT_MODEL = OLLAMA_CHAT_MODEL` (saves a resident model at the cost of slower routing when the chat model is large).
7. Narrow `llm_thinking_enabled` to router/planner only, not every context.
8. Reduce `intent_judge_timeout_sec` (15s) or race it against text-based wake detection to avoid blocking the audio loop.

View File

@@ -85,6 +85,12 @@ class Settings:
llm_digest_timeout_sec: float
llm_embedding_timeout_sec: float
llm_profile_select_timeout_sec: float
# Upper bound on tokens the chat model may generate per reply turn. Spoken
# (TTS) answers are 1-2 sentences, so a cap bounds the worst-case latency of
# a model that occasionally rambles or loops without changing normal answers.
# The headroom (default 512) sits well above this app's short tool-call JSON,
# so capping never truncates a tool call. 0 (or negative) disables the cap.
ollama_num_predict: int
# Profiles & Behavior
active_profiles: list[str]
@@ -394,6 +400,9 @@ def get_default_config() -> Dict[str, Any]:
"llm_digest_timeout_sec": 8.0,
"llm_embedding_timeout_sec": 60.0,
"llm_profile_select_timeout_sec": 30.0,
# Cap on chat-model output tokens per turn (worst-case latency guard).
# 512 is safe headroom above short TTS answers and tool-call JSON; 0 off.
"ollama_num_predict": 512,
# Profiles & Behavior
"active_profiles": ["developer", "business", "life"],
@@ -608,7 +617,11 @@ def load_settings() -> Settings:
active_profiles = _ensure_list(merged.get("active_profiles"))
tts_enabled = bool(merged.get("tts_enabled", True))
tts_engine = str(merged.get("tts_engine", "piper")).lower()
if tts_engine not in ("piper", "chatterbox"):
# "edge" (Microsoft Edge TTS) is the containerized bridge's Korean voice;
# "melo" is the legacy warm-worker voice. Both are multilingual, so they must
# be preserved here — coercing them to "piper" would mislabel the engine as
# English-only in reply_language_directive().
if tts_engine not in ("piper", "chatterbox", "edge", "melo"):
tts_engine = "piper" # Default to piper if invalid value
tts_voice_val = merged.get("tts_voice")
tts_voice = None if tts_voice_val in (None, "", "null") else str(tts_voice_val)
@@ -759,6 +772,10 @@ def load_settings() -> Settings:
llm_digest_timeout_sec = float(merged.get("llm_digest_timeout_sec", 8.0))
llm_embedding_timeout_sec = float(merged.get("llm_embedding_timeout_sec", 60.0))
llm_profile_select_timeout_sec = float(merged.get("llm_profile_select_timeout_sec", 30.0))
try:
ollama_num_predict = int(merged.get("ollama_num_predict", 512))
except (TypeError, ValueError):
ollama_num_predict = 512
return Settings(
# Database & Storage
@@ -774,6 +791,7 @@ def load_settings() -> Settings:
llm_digest_timeout_sec=llm_digest_timeout_sec,
llm_embedding_timeout_sec=llm_embedding_timeout_sec,
llm_profile_select_timeout_sec=llm_profile_select_timeout_sec,
ollama_num_predict=ollama_num_predict,
# Profiles & Behavior
active_profiles=active_profiles,

View File

@@ -2233,6 +2233,16 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
has_tool_calls = " (has tool_calls)" if msg.get("tool_calls") else ""
debug_log(f" [{i}] {role}: {content}{has_tool_calls}", "planning")
# Bound worst-case generation latency: spoken answers are 1-2 sentences,
# so cap the chat model's output tokens. The default headroom sits well
# above this app's tool-call JSON, so capping never truncates a tool
# call. 0/negative disables it. See config.ollama_num_predict.
try:
_num_predict = int(getattr(cfg, 'ollama_num_predict', 0) or 0)
except (TypeError, ValueError):
_num_predict = 0
_chat_extra_options = {"num_predict": _num_predict} if _num_predict > 0 else None
# Send messages to Ollama — try native tool calling first, fall back to text-based
# if the model returns HTTP 400 (native tools API not supported).
_dump_tools_schema = None if use_text_tools else tools_json_schema
@@ -2242,7 +2252,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
chat_model=cfg.ollama_chat_model,
messages=messages,
timeout_sec=float(getattr(cfg, 'llm_chat_timeout_sec', 45.0)),
extra_options=None,
extra_options=_chat_extra_options,
tools=_dump_tools_schema,
thinking=getattr(cfg, 'llm_thinking_enabled', False),
)
@@ -2273,7 +2283,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
chat_model=cfg.ollama_chat_model,
messages=messages,
timeout_sec=float(getattr(cfg, 'llm_chat_timeout_sec', 45.0)),
extra_options=None,
extra_options=_chat_extra_options,
tools=None,
thinking=getattr(cfg, 'llm_thinking_enabled', False),
)

View File

@@ -22,11 +22,15 @@ class ModelSize(Enum):
LARGE = "large" # 8b+ - can infer tool usage from context
# Model size patterns - models matching these are considered SMALL
# Model size patterns - models matching these are considered SMALL.
# Covers every sub-8B size (1b-7b): these models need the explicit, repeated
# tool/greeting/instruction constraints and falter on the terse LARGE prompt.
# Without 2b/4b/5b/6b here a genuinely small model (e.g. qwen*:4b) was
# misclassified as LARGE and given the less-guided prompt set.
_SMALL_MODEL_PATTERNS = (
":1b", ":3b", ":7b",
"-1b", "-3b", "-7b",
"_1b", "_3b", "_7b",
":1b", ":2b", ":3b", ":4b", ":5b", ":6b", ":7b",
"-1b", "-2b", "-3b", "-4b", "-5b", "-6b", "-7b",
"_1b", "_2b", "_3b", "_4b", "_5b", "_6b", "_7b",
"gemma4", # Gemma 4 - always small regardless of tag
)

View File

@@ -4,7 +4,7 @@ This module provides model-size-aware prompt generation for the reply engine.
### Problem Statement
Small models (1b, 3b, 7b parameters) lack the reasoning capacity to infer when NOT to use tools. When given prompts like "Proactively use available tools," they may incorrectly call tools for simple greetings like "hello" or "ni hao" because they cannot distinguish between:
Small models (every sub-8B size, 1b-7b parameters) lack the reasoning capacity to infer when NOT to use tools. When given prompts like "Proactively use available tools," they may incorrectly call tools for simple greetings like "hello" or "ni hao" because they cannot distinguish between:
- Requests that require tools (weather, search, data retrieval)
- Simple conversation (greetings, small talk, general knowledge)
@@ -14,7 +14,7 @@ The module detects model size from the model name and selects appropriate prompt
| Model Size | Detection Pattern | Tool Prompts |
|------------|-------------------|--------------|
| SMALL | `:1b`, `:3b`, `:7b`, `gemma4` | Conservative — explicit "DO NOT use tools for greetings" + worked negative examples + repetition |
| SMALL | `:1b`-`:7b` (every size 1-7B, all separators), `gemma4` | Conservative — explicit "DO NOT use tools for greetings" + worked negative examples + repetition |
| LARGE | All others (8b+) | Proactive — "use tools confidently" + short anti-confabulation + auto-derive clause |
### Architecture
@@ -43,7 +43,7 @@ from jarvis.reply.prompts import (
Both model sizes share these base components:
- `asr_note`: Voice transcription error handling
- `inference_guidance`: Prefer inference over clarification
- `voice_style`: Concise, conversational responses
- `voice_style`: Single-sentence, conversational responses (spoken aloud, so one sentence only — never more)
Model-size-specific components:
- `tool_incentives`: When/how aggressively to use tools

View File

@@ -26,8 +26,8 @@ INFERENCE_GUIDANCE = (
# Voice assistant communication style - concise, conversational
VOICE_STYLE = (
"Keep responses concise and conversational since this is a voice assistant. "
"Two to three sentences maximum. Prioritize clarity and brevity - users are listening, not reading. "
"Avoid unnecessary elaboration unless specifically requested. "
"Reply in a SINGLE sentence - never more than one sentence. Prioritize clarity and brevity - users are listening, not reading. "
"Avoid unnecessary elaboration. "
"Do NOT offer follow-up suggestions or ask if the user wants more info - just respond directly. "
"IMPORTANT: Always respond in natural language - never output JSON, code, or structured data as your response. "
"NEVER use markdown formatting in your replies: no asterisks for emphasis (**bold**, *italic*), "

View File

@@ -287,6 +287,8 @@ Turn 4: LLM → {content: "Here's a comprehensive comparison of the iPhone 15 mo
- `llm_tools_timeout_sec` (enrichment extraction)
- `llm_embed_timeout_sec` (vector search)
- `llm_chat_timeout_sec` (messages loop turn)
- Output bound:
- `ollama_num_predict` (default `512`, `0`/negative disables) caps the chat model's generated tokens per turn via the Ollama `num_predict` option on the messages-loop call. Spoken (TTS) answers are 1-2 sentences, so this never clips a normal answer; it bounds the worst-case latency of a model that occasionally rambles or loops. The default headroom sits well above this app's short tool-call JSON, so it does not truncate tool calls. Applied uniformly to the reply loop's chat call (both native-tools and text-tools paths); the small classification passes (intent judge, digests) keep their own caps. Note: this is a worst-case guard, not the primary latency lever, which is model size and GPU residency.
- Memory enrichment:
- `memory_enrichment_max_results` limits recalled snippets.
- `memory_digest_enabled` (default `null` = auto-on for SMALL models ≤7B, off for LARGE) distils the combined diary + graph dump into a short relevance-filtered note via a cheap LLM pass before injecting into the system prompt. See **Memory Digest for Small Models** below.

View File

@@ -62,10 +62,11 @@ _SYSTEM_PROMPT_TEMPLATE: str = (
"Tone rails (hard): never mean, never condescending, never passive-aggressive, never "
"sulking, never preachy, never sycophantic ('great question', 'I'd be happy to'). "
"Sarcasm points at the situation, the topic, or mildly at yourself — never at the user. "
"Shape for casual, factual, or small-talk replies: state the answer in a sentence, then add "
"one short dry observation about it (an understated aside, a raised-eyebrow remark, a gentle "
"noticing of the irony). One aside — not two, not a joke opener, not a joke-shaped sentence "
"replacing the answer. The aside is a tail, not the head. "
"Shape for casual, factual, or small-talk replies: give the answer in a SINGLE sentence. If a "
"dry aside fits, fold it into that same sentence as a short trailing clause — never add it as "
"a second sentence, never a joke opener, never a joke-shaped sentence replacing the answer. "
"Whenever the wit would require a second sentence, drop the wit and keep the one-sentence "
"answer. The aside is a tail inside the sentence, not a head and not a new sentence. "
"Examples of the MOVE (shape, not wording — never copy these): stating a fact and then noting "
"its mild absurdity; giving the weather and then commenting on what it implies for the day; "
"answering a trivia question and then offering a wry footnote about the subject; admitting "
@@ -79,8 +80,8 @@ _SYSTEM_PROMPT_TEMPLATE: str = (
"butler clichés, and never address the user as 'sir', 'madam', 'my liege', or similar. "
"Never stack multiple jokes in one reply. "
"Be concise, conversational, and actionable. "
"This is a spoken voice assistant: answer in ONE short sentence whenever possible "
"(two at the very most). No lists, no preamble, no 'is there anything else' offers. "
"This is a spoken voice assistant: your ENTIRE reply must be a single short sentence. "
"Never write a second sentence. No lists, no preamble, no 'is there anything else' offers. "
"When a controlBrowser tool is available, use IT (never webSearch) for anything that "
"should happen in the on-screen browser — opening a site, searching on a site "
"(controlBrowser action 'search' with the right site), clicking, typing — because only "
@@ -119,10 +120,13 @@ _SYSTEM_PROMPT_TEMPLATE: str = (
"'tell me a joke', 'chat with me'), never reply with a bare greeting like 'Hey there!', "
"'Hi!', 'How can I help you?', or a generic observation about an unrelated topic. "
"When the 'Information the user has shared…' section is present, you MUST pick one concrete "
"fact from it and build the reply around that fact (e.g. 'You mentioned you box at Trenches "
"Gym — how's training going this week?'). Do not talk about things that are not in that "
"section. Only when that section is absent may you invent a fresh observation, question, or "
"joke. Produce a varied response each time — do not repeat a previous reply verbatim. "
"fact from it and build the reply around that fact, opening with a short natural reference to "
"it. CRITICAL: use ONLY names, places, activities, and details that literally appear in that "
"section — never borrow any name, place, or activity from these instructions or from any "
"example wording, and never invent specifics that are not in that section. Do not talk about "
"things that are not in that section. Only when that section is absent may you invent a fresh "
"observation, question, or joke. Produce a varied response each time — do not repeat a "
"previous reply verbatim. "
"Banned phrasings: 'I can only tell you what you have shared with me in this conversation', "
"'I don't have access to any personal information outside of what you tell me', 'I don't have "
"personal details outside of our conversation history', 'I do not store personal details "

View File

@@ -30,8 +30,10 @@ class BrowseAndPlayTool(Tool):
def description(self) -> str:
return (
"Play a song / music video / clip on the shared screen by searching YouTube "
"and playing the first result. Use when the user asks you to play or watch "
"something. Only available in screen-share mode."
"and playing a result. Use when the user asks you to play or watch "
"something. Plays the first result by default; pass 'index' to play the "
"Nth result from the top of the search list (e.g. 'play the 3rd video' -> "
"index=3). Only available in screen-share mode."
)
@property
@@ -42,7 +44,16 @@ class BrowseAndPlayTool(Tool):
"query": {
"type": "string",
"description": "What to play, e.g. 'IU Good Day' or 'lofi hip hop'.",
}
},
"index": {
"type": "integer",
"description": (
"1-based position of the video to play in the search results, "
"counted from the top of the list. Defaults to 1 (first result). "
"Use for 'play the Nth video' / 'play the second one'."
),
"minimum": 1,
},
},
"required": ["query"],
}
@@ -55,18 +66,25 @@ class BrowseAndPlayTool(Tool):
reply_text="화면 공유 모드(STREAM_BROWSER=true)에서만 영상을 재생할 수 있습니다.",
)
query = ""
index = 1
if args and isinstance(args, dict):
query = str(args.get("query", "")).strip()
try:
index = int(args.get("index", 1) or 1)
except (TypeError, ValueError):
index = 1
if index < 1:
index = 1
if not query:
return ToolExecutionResult(success=False, reply_text="재생할 내용을 알려주세요.")
if not _NODE_SCRIPT.exists():
return ToolExecutionResult(success=False, reply_text="브라우저 재생 도구를 찾을 수 없습니다.")
context.user_print(f"▶️ 화면에서 '{query}' 재생 중…")
debug_log(f" ▶️ browseAndPlay '{query}'", "tools")
context.user_print(f"▶️ 화면에서 '{query}' 재생 중… (#{index})")
debug_log(f" ▶️ browseAndPlay '{query}' index={index}", "tools")
try:
proc = subprocess.run(
["node", str(_NODE_SCRIPT), query, "youtube"],
["node", str(_NODE_SCRIPT), query, "youtube", str(index)],
capture_output=True,
text=True,
timeout=40,

View File

@@ -6,16 +6,24 @@ video, or clip.
### Behaviour
- Public schema is a single required `query` string (what to play).
- Public schema is a required `query` string (what to play) plus an optional
`index` integer (1-based position in the search results, counted from the top
of the list). `index` defaults to `1` (first result), so existing callers and
"play X" requests are unchanged; "play the 3rd video" / "play the second one"
map to `index=3` / `index=2`.
- **Mode-gated**: only acts when `STREAM_BROWSER` is true (`cfg.stream_browser`).
In voice-only mode (false) there is no screen to show, so it returns a short
message and does nothing.
- Drives the on-screen Chrome by subprocessing the Node CDP helper
`bot/scripts/stream-test/browse-search.mjs <query> youtube`, which searches
YouTube and plays the first result on display `:1`. The broadcast captures
that display, so the playback is what viewers see.
- Returns `success` with the played video's title, or a failure message if the
helper/Chrome is unavailable. It does NOT make an LLM call.
`bot/scripts/stream-test/browse-search.mjs <query> youtube <index>`, which
searches YouTube and plays the chosen result on display `:1`. The broadcast
captures that display, so the playback is what viewers see.
- The helper clicks the `index`-th `a#video-title` in the results list. The
index is clamped to the number of results actually returned, so asking for a
position beyond the list plays the last available result rather than failing.
- Returns `success` with the played video's title (and the resolved `index`), or
a failure message if the helper/Chrome is unavailable. It does NOT make an LLM
call.
### Principles

View File

@@ -45,9 +45,16 @@ class ControlBrowserTool(Tool):
"Use this (NOT webSearch) whenever the user wants something done or shown IN the "
"browser on screen: open a website or URL, search on a specific site (action "
"'search' with site=naver/google/daum/youtube/bing), go back/forward, refresh, "
"manage tabs (list/new/close/switch), close popups, click, type, scroll, or "
"screenshot. webSearch only returns text and shows nothing on screen; this tool "
"actually navigates the visible browser. Only available in screen-share mode. "
"manage tabs (list/new/close/switch), close popups, move the mouse onto an element, "
"click, type, scroll, or screenshot. webSearch only returns text and shows nothing "
"on screen; this tool actually navigates the visible browser. "
"Cursor behaviour matters: 'search' and 'type' (with a selector) move the REAL mouse "
"cursor to the on-page box, click it, then type one character at a time — use these "
"when the user wants to search or type on a page. 'navigate' only types the URL into "
"the address bar (no mouse movement). 'moveMouse' moves/hovers the visible cursor "
"onto an element (selector, or site=naver/... for that site's search box) WITHOUT "
"clicking — use it when the user just asks to move the mouse somewhere. "
"Only available in screen-share mode. "
"Never claim you did any of this unless this tool returns success."
)
@@ -61,16 +68,16 @@ class ControlBrowserTool(Tool):
"enum": [
"status", "listTabs", "navigate", "search", "back",
"forward", "refresh", "newTab", "closeTab", "activateTab",
"closePopups", "click", "type", "scroll", "pressKey",
"closePopups", "moveMouse", "click", "type", "scroll", "pressKey",
"screenshot",
],
"description": "What to do in the browser.",
},
"url": {"type": "string", "description": "Target URL/site for navigate/newTab (e.g. 'naver.com')."},
"query": {"type": "string", "description": "Search text for action 'search'."},
"site": {"type": "string", "description": "Search site for action 'search': naver, google, daum, youtube, bing."},
"site": {"type": "string", "description": "Site for action 'search' or 'moveMouse': naver, google, daum, youtube, bing. For moveMouse it targets that site's search box."},
"index": {"type": "integer", "description": "Tab index for closeTab/activateTab (from listTabs)."},
"selector": {"type": "string", "description": "CSS selector for click/type."},
"selector": {"type": "string", "description": "CSS selector for click/type/moveMouse."},
"text": {"type": "string", "description": "Text to type."},
"key": {"type": "string", "description": "Key to press, e.g. 'Return', 'Escape'."},
"dir": {"type": "string", "description": "Scroll direction: 'down' or 'up'."},
@@ -149,7 +156,12 @@ class ControlBrowserTool(Tool):
if action == "navigate":
return f"브라우저에서 {data.get('url', args.get('url'))} 로 이동했습니다."
if action == "search":
return f"{data.get('site', '')}에서 '{data.get('query', args.get('query'))}'를 검색해 화면에 띄웠습니다."
base = f"{data.get('site', '')}에서 '{data.get('query', args.get('query'))}'를 검색해 화면에 띄웠습니다."
# Flag when the real cursor path didn't run, so a silent fallback to
# cursor-less DOM input is visible rather than reported as "human".
if data.get("input") in ("api", "api-fallback"):
base += " (참고: 실제 마우스 커서 이동 없이 처리됨)"
return base
if action in ("back", "forward", "refresh"):
return f"브라우저: {action} 완료 ({data.get('url', '')})."
if action in ("status", "listTabs"):
@@ -164,6 +176,9 @@ class ControlBrowserTool(Tool):
return f"{data.get('active')}번으로 전환했습니다."
if action == "closePopups":
return f"팝업/빈 탭 {data.get('closed')}개를 닫았습니다."
if action == "moveMouse":
target = data.get("target") or args.get("selector") or args.get("site") or "대상"
return f"마우스 커서를 {target} 위치로 옮겼습니다."
if action == "screenshot":
return f"화면을 캡처했습니다: {data.get('path')}"
return "완료했습니다."

View File

@@ -0,0 +1,79 @@
"""Tests for browseAndPlay's ``index`` argument (play the Nth search result).
Behaviour verified:
- default plays the first result (index 1) and stays backward-compatible,
- an explicit index is forwarded to the Node helper as the 4th argv,
- bad / sub-1 index values clamp to 1,
- the index is advertised in the tool schema.
"""
import json
from unittest.mock import Mock, patch
import pytest
from src.jarvis.tools.builtin.browse_and_play import BrowseAndPlayTool, _NODE_SCRIPT
def _ctx():
cfg = Mock()
cfg.stream_browser = True
return Mock(cfg=cfg, user_print=Mock())
def _run(args):
tool = BrowseAndPlayTool()
with patch("src.jarvis.tools.builtin.browse_and_play.subprocess.run") as mock_run:
mock_run.return_value = Mock(
stdout=json.dumps({"ok": True, "title": "Some Video"}),
stderr="",
)
result = tool.run(args, _ctx())
return mock_run, result
def _argv(mock_run):
return list(mock_run.call_args[0][0])
@pytest.mark.unit
def test_schema_exposes_index():
schema = BrowseAndPlayTool().inputSchema
assert "index" in schema["properties"]
assert schema["properties"]["index"]["type"] == "integer"
assert "query" in schema["required"]
assert "index" not in schema["required"] # optional
@pytest.mark.unit
def test_default_index_is_first():
mock_run, result = _run({"query": "IU Good Day"})
argv = _argv(mock_run)
assert argv[:4] == ["node", str(_NODE_SCRIPT), "IU Good Day", "youtube"]
assert argv[4] == "1"
assert result.success is True
@pytest.mark.unit
def test_explicit_index_forwarded():
mock_run, _ = _run({"query": "lofi", "index": 3})
assert _argv(mock_run)[4] == "3"
@pytest.mark.unit
@pytest.mark.parametrize("bad", [0, -2, "nope", None])
def test_bad_index_clamps_to_one(bad):
mock_run, _ = _run({"query": "lofi", "index": bad})
assert _argv(mock_run)[4] == "1"
@pytest.mark.unit
def test_voice_only_mode_does_not_play():
tool = BrowseAndPlayTool()
cfg = Mock()
cfg.stream_browser = False
ctx = Mock(cfg=cfg, user_print=Mock())
with patch("src.jarvis.tools.builtin.browse_and_play.subprocess.run") as mock_run:
result = tool.run({"query": "x", "index": 2}, ctx)
assert result.success is False
mock_run.assert_not_called()

View File

@@ -0,0 +1,56 @@
"""Tests for the controlBrowser tool's action surface.
These are deterministic schema/summary checks — they do not drive a real
browser. The actual cursor movement is exercised live on the browser host
(xdotool + CDP), which these tests cannot reach.
"""
import pytest
from jarvis.tools.builtin.control_browser import ControlBrowserTool
@pytest.fixture
def tool():
return ControlBrowserTool()
def test_movemouse_is_an_exposed_action(tool):
# A weak model confabulated "moved the mouse" because no move/hover action
# existed to call. The cursor-move capability must be a real action so the
# request "move the mouse to the search box" maps to a tool call.
enum = tool.inputSchema["properties"]["action"]["enum"]
assert "moveMouse" in enum
def test_movemouse_summary_reports_the_target(tool):
summary = tool._summarise("moveMouse", {"site": "naver"}, {"ok": True, "target": "naver"})
assert "마우스" in summary and "naver" in summary
def test_description_distinguishes_cursor_paths(tool):
# The model must know navigate is address-bar only (no mouse) while
# search/type/moveMouse move the real cursor — that distinction is the
# whole point of the fix.
desc = tool.description
assert "moveMouse" in desc
assert "address bar" in desc # navigate is described as address-bar typing
def test_search_summary_flags_cursorless_fallback(tool):
# When the real xdotool cursor path didn't run, the summary must say so
# rather than implying a human-like search happened.
human = tool._summarise("search", {"query": "날씨"}, {"ok": True, "site": "naver", "query": "날씨", "input": "human"})
assert "참고: 실제 마우스" not in human
fell_back = tool._summarise("search", {"query": "날씨"}, {"ok": True, "site": "naver", "query": "날씨", "input": "api-fallback"})
assert "실제 마우스 커서 이동 없이" in fell_back
def test_movemouse_summary_only_runs_on_success(tool):
# _summarise is only called on ok:true; an ok:false (target not found / no
# xdotool) is handled by run() as a failure reply, so a failed move can no
# longer be reported as "moved". Sanity-check the success summary names a
# target rather than a placeholder when one is present.
summary = tool._summarise("moveMouse", {"selector": "#query"}, {"ok": True, "target": "#query"})
assert "#query" in summary

View File

@@ -0,0 +1,62 @@
"""The docker deployment must run auxiliary calls on a small model.
Latency win: intent judging, tool routing and arg extraction are
classification/JSON calls, not the spoken answer. Running them on a small fast
model means the big chat model only runs once per command (for the answer),
instead of 2-3 times per command for routing/extraction too.
The wiring is: docker/jarvis-config.template.json renders `intent_judge_model`
from `${OLLAMA_INTENT_MODEL}`, and the reply engine's resolver falls through
`tool_router_model -> intent_judge_model -> ollama_chat_model`. The template
sets no `tool_router_model`, so the auxiliary model is whatever
`OLLAMA_INTENT_MODEL` renders to. These tests pin that behaviour end to end.
"""
import json
import string
from pathlib import Path
import pytest
from jarvis.reply.engine import resolve_tool_router_model
TEMPLATE = Path(__file__).resolve().parent.parent / "docker" / "jarvis-config.template.json"
def _render(**env) -> dict:
raw = TEMPLATE.read_text(encoding="utf-8")
return json.loads(string.Template(raw).safe_substitute(**env))
class _Cfg:
"""cfg stand-in carrying only the fields the resolver reads. The template
does not render `tool_router_model`, so it stays empty here too."""
def __init__(self, rendered: dict):
self.tool_router_model = rendered.get("tool_router_model", "") or ""
self.intent_judge_model = rendered.get("intent_judge_model", "") or ""
self.ollama_chat_model = rendered.get("ollama_chat_model", "") or ""
def test_template_renders_separate_intent_model():
cfg = _render(OLLAMA_CHAT_MODEL="qwen3:8b", OLLAMA_INTENT_MODEL="qwen2.5:3b")
assert cfg["ollama_chat_model"] == "qwen3:8b"
assert cfg["intent_judge_model"] == "qwen2.5:3b"
assert cfg["intent_judge_model"] != cfg["ollama_chat_model"]
@pytest.mark.unit
def test_aux_calls_route_to_small_model_not_chat_model():
"""The whole point: with a big chat model and a small intent model, tool
routing must resolve to the small model, leaving the big model for answers."""
cfg = _Cfg(_render(OLLAMA_CHAT_MODEL="qwen3:8b", OLLAMA_INTENT_MODEL="qwen2.5:3b"))
assert resolve_tool_router_model(cfg) == "qwen2.5:3b"
@pytest.mark.unit
def test_folding_intent_onto_chat_model_keeps_one_model():
"""Setting OLLAMA_INTENT_MODEL == OLLAMA_CHAT_MODEL folds everything back
onto a single resident model (the documented VRAM-saving opt-out)."""
cfg = _Cfg(_render(OLLAMA_CHAT_MODEL="qwen2.5:3b", OLLAMA_INTENT_MODEL="qwen2.5:3b"))
assert resolve_tool_router_model(cfg) == "qwen2.5:3b"
assert cfg.intent_judge_model == cfg.ollama_chat_model

View File

@@ -0,0 +1,112 @@
"""Tests for the ``ollama_num_predict`` chat-output cap.
The cap bounds worst-case reply latency by limiting how many tokens the chat
model may generate per turn. Spoken (TTS) answers are 1-2 sentences, so the
default headroom never clips a normal answer and stays above tool-call JSON.
These tests verify behaviour:
- the config default is present,
- the value is threaded into the Ollama request as the ``num_predict`` option,
- the reply loop forwards it to the chat call (and disables it at 0).
"""
from unittest.mock import Mock, patch
import pytest
from src.jarvis.config import get_default_config
from src.jarvis.memory.conversation import DialogueMemory
from src.jarvis.reply.engine import run_reply_engine
# ---------------------------------------------------------------------------
# Config default
# ---------------------------------------------------------------------------
def test_default_config_has_num_predict_cap():
config = get_default_config()
assert config["ollama_num_predict"] == 512
# ---------------------------------------------------------------------------
# Transport: extra_options.num_predict reaches the Ollama payload options
# ---------------------------------------------------------------------------
@patch("jarvis.llm.requests.post")
def test_chat_with_messages_forwards_num_predict(mock_post):
from jarvis.llm import chat_with_messages
mock_resp = Mock()
mock_resp.status_code = 200
mock_resp.json.return_value = {"message": {"content": "ok"}}
mock_resp.raise_for_status = Mock()
mock_post.return_value = mock_resp
chat_with_messages(
"http://localhost:11434",
"test-large",
[{"role": "user", "content": "hi"}],
extra_options={"num_predict": 512},
)
_, kwargs = mock_post.call_args
options = (kwargs.get("json") or {}).get("options") or {}
assert options.get("num_predict") == 512
# ---------------------------------------------------------------------------
# Reply loop wiring
# ---------------------------------------------------------------------------
def _mock_cfg(num_predict):
cfg = Mock()
cfg.ollama_base_url = "http://localhost:11434"
cfg.ollama_chat_model = "test-large" # avoid SMALL-model text-tool path
cfg.ollama_num_predict = num_predict
cfg.voice_debug = False
cfg.llm_tools_timeout_sec = 8.0
cfg.llm_embed_timeout_sec = 10.0
cfg.llm_chat_timeout_sec = 45.0
cfg.llm_digest_timeout_sec = 8.0
cfg.memory_enrichment_max_results = 5
cfg.memory_enrichment_source = "diary"
cfg.memory_digest_enabled = False
cfg.tool_result_digest_enabled = False
cfg.location_ip_address = None
cfg.location_auto_detect = False
cfg.location_enabled = False
cfg.agentic_max_turns = 8
cfg.tool_search_max_calls = 3
cfg.tool_selection_strategy = "all"
cfg.tool_carryover_max_turns = 2
cfg.tool_carryover_per_entry_chars = 1200
cfg.mcps = {}
cfg.llm_thinking_enabled = False
cfg.tts_engine = "none"
cfg.ollama_embed_model = "test-embed"
return cfg
def _run_single_turn(cfg):
"""Drive one reply turn that answers in plain text and capture the
chat call's extra_options."""
with patch("src.jarvis.reply.engine.plan_query", return_value=[]), \
patch("src.jarvis.reply.engine.extract_search_params_for_memory", return_value={}), \
patch("src.jarvis.reply.engine.extract_text_from_response", return_value="Hello."), \
patch("src.jarvis.reply.engine.chat_with_messages") as mock_chat:
mock_chat.return_value = {"message": {"content": "Hello."}}
run_reply_engine(db=Mock(), cfg=cfg, tts=None,
text="hi", dialogue_memory=DialogueMemory())
assert mock_chat.called
return mock_chat.call_args.kwargs.get("extra_options")
@pytest.mark.unit
def test_reply_loop_caps_output_when_enabled():
extra = _run_single_turn(_mock_cfg(512))
assert extra == {"num_predict": 512}
@pytest.mark.unit
def test_reply_loop_no_cap_when_zero():
extra = _run_single_turn(_mock_cfg(0))
assert extra is None

View File

@@ -21,9 +21,15 @@ class TestModelSizeDetection:
("gemma:7b", True),
("phi3:3b", True),
("qwen2:7b", True),
# Sub-8B sizes that were previously misclassified as LARGE.
("qwen3.5:4b", True), # the deployed model that produced weak, off-tone replies
("gemma2:2b", True),
("model:5b", True),
("model:6b", True),
# Various separators
("model-3b-instruct", True),
("model_1b_chat", True),
("model-4b-instruct", True),
# Large models (should return LARGE)
("gpt-oss:20b", False),
("llama3.1:8b", False),
@@ -121,6 +127,18 @@ class TestPromptComponents:
assert prompts.voice_style, f"{size.value} missing voice_style"
assert prompts.tool_guidance, f"{size.value} missing tool_guidance"
def test_voice_style_enforces_single_sentence(self):
"""voice_style must cap replies at one sentence (spoken aloud). The old
'Two to three sentences maximum' wording let the model run long, which
also slowed TTS since synth time scales with text length."""
from jarvis.reply.prompts import get_system_prompts, ModelSize
for size in [ModelSize.SMALL, ModelSize.LARGE]:
voice_style = get_system_prompts(size).voice_style
assert "SINGLE sentence" in voice_style, f"{size.value} voice_style not single-sentence"
assert "never more than one sentence" in voice_style
assert "Two to three" not in voice_style
def test_to_list_returns_non_empty_strings(self):
"""to_list() returns only non-empty prompt strings."""
from jarvis.reply.prompts import get_system_prompts, ModelSize

View File

@@ -44,6 +44,32 @@ class TestBuildSystemPrompt:
assert "in the user's language" not in prompt
assert "in Korean" in prompt
def test_persona_enforces_single_sentence(self):
# Spoken replies must be one sentence (TTS latency scales with text
# length, and the user asked for one-sentence answers). The persona must
# state the single-sentence rule and must NOT carry the old "two at the
# very most" allowance that let the model run long.
prompt = build_system_prompt("Jarvis")
assert "single short sentence" in prompt
assert "Never write a second sentence" in prompt
assert "two at the very most" not in prompt
def test_persona_aside_does_not_authorise_a_second_sentence(self):
# The dry aside must fold into the one sentence, not become a 2nd one.
prompt = build_system_prompt("Jarvis")
assert "SINGLE sentence" in prompt
assert "never add it as " in prompt
def test_persona_has_no_copyable_proper_noun_examples(self):
# A weak model parroted the literal "Trenches Gym" example from the
# persona as if it were a real user fact (boxing mangled to tennis).
# The persona must not embed copyable personal proper nouns, and must
# tell the model to use ONLY facts that literally appear in the memory
# section — never borrow names/places from the instructions themselves.
prompt = build_system_prompt("Jarvis")
assert "Trenches" not in prompt
assert "never borrow any name, place, or activity from these instructions" in prompt
class TestOutputLanguageDirective:
"""A deployment may lock replies to a single language via OUTPUT_LANGUAGE.
@@ -110,6 +136,14 @@ class TestReplyLanguageDirective:
directive = reply_language_directive("Korean", "melo")
assert directive is not None and "Korean" in directive
def test_edge_is_multilingual(self):
# Edge TTS (the default Korean voice) is not English-only: no lock → the
# user's own language, and a lock is honoured (not forced to English).
assert reply_language_directive(None, "edge") is None
directive = reply_language_directive("Korean", "edge")
assert directive is not None and "Korean" in directive
assert directive != ENGLISH_ONLY_DIRECTIVE
class TestLoadAgentInstructions:
"""Operator can extend the reply LLM's system prompt by dropping *.md files