Empirical A/B/C measurement against the live RTX 5050 Ollama stack
(qwen2.5:3b + nomic-embed-text) showed keep_alive=0 unloads the embed
model ~2s after every call, so each turn after a brief idle gap pays a
cold reload. VRAM is not the constraint (~4.4-4.7 GB free with both
models resident) and keep_alive=0 never evicted the chat model, so CPU
embedding (num_gpu=0) gave no benefit. A short positive keep_alive is
the fastest of the three: it keeps the ~0.3 GB embed model resident
across consecutive turns at negligible VRAM cost.
Add tests/test_embeddings.py covering the warm-across-turns behaviour.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Open-Meteo's geocoder only matches Latin/English spellings, so a Korean city
name like "서울" returns zero results even though the place exists. With
OUTPUT_LANGUAGE locked to Korean the tool-calling model naturally fills
`location` with the Korean name, which dead-ended on every weather request
("could not find location").
When the first geocode is empty and the name is non-ASCII, ask the warm small
model for the common English exonym ("서울" -> "Seoul") and retry once. ASCII
names skip the round-trip entirely.
Live-verified: "서울 날씨 알려줘" now returns real Seoul weather. Tests cover the
romanise-and-retry path and the ASCII short-circuit.
qwen2.5:3b emits tool calls in text shapes the parser dropped, breaking
two reviewer-reported behaviours:
- `getWeather: {"location": "Seoul"}` (a JSON object after the colon) was
dumped wholesale into {"query": "{...}"}, so `location` never reached the
tool. getWeather then ran with empty args, returned the auto-detected
location's weather, the model noticed the mismatch and retried — looping up
to 8 times before giving up with an English error. Now the JSON object after
the colon is parsed directly as the argument dict.
- `call_stop: {"id":..., "function": {"name": "setBroadcast",
"arguments": "{\"action\": \"stop\"}"}}` — a single tool_call object without
the `tool_calls: [...]` array wrapper, behind a `call_xxx:` label — matched
no form, so the raw JSON leaked to the user AND setBroadcast never ran
("방송 꺼줘" did nothing). Now name + arguments are pulled from the embedded
`function` object when the name is in the allow-list.
Field-captured from the live qwen2.5:3b brain (2026-06-12). Tests cover both
shapes, non-ASCII args, dict/string arguments, and unknown-tool rejection.
Harden the reply-language lock so qwen2.5:3b reliably stays in the locked
language instead of leaking the query language back in:
- reply_language_directive(): single resolver with clear precedence —
explicit OUTPUT_LANGUAGE lock wins over the Piper/Chatterbox English-only
fallback (this deployment's actual TTS is Korean MeloTTS, so the legacy
English lock was both wrong and contradicting the Korean lock).
- Stronger, override-explicit directive wording, inserted near the FRONT of
the system prompt so a small model gives it primacy over the persona.
- build_system_prompt(output_language=...): rewrite the persona's "in the
user's language" clause to the locked language so the persona stops
fighting the lock.
- docs/llm_contexts.md: document the resolver, precedence, and placement.
Live-verified on the running brain (qwen2.5:3b): Korean voice-style input
and a cold English query both return fully Korean replies with no CJK/Hanja
leak. Tests cover unset/set/agnostic/whitespace + precedence + persona rewrite.
Add an optional OUTPUT_LANGUAGE env var that forces every reply into a
single language. When set, output_language_directive() injects a "respond
only in <language>" instruction (also forbidding other scripts) into the
chat loop's system prompt, next to the existing TTS English-only lock.
Empty (default) keeps the multilingual "reply in the user's language"
behaviour, so upstream is unaffected.
For the Korean-only deployment this also suppresses the occasional trailing
CJK/Hanja fragment qwen2.5:3b leaks on free-form chit-chat.
- system_prompt.py: language-agnostic output_language_directive() helper
- engine.py: read OUTPUT_LANGUAGE, append directive in _build_initial_system_message
- docker-compose.yml + .env.example: document/pass the new var
- docs/llm_contexts.md: note the new gating on the main reply context
- tests: cover unset/set/agnostic/whitespace cases
Completes the STREAM_BROWSER=true behaviour:
- handleJoin auto-starts the broadcast on voice join and wires the session to
the guild streamer; each turn reports the live state to the brain so search
routes Chrome (live) vs Gemini (off).
- New setBroadcast tool lets the user toggle the broadcast by voice ("방송
켜줘/꺼줘") via the LLM (no hardcoded phrases); it refuses when
STREAM_BROWSER=false. The directive flows brain -> bridge (broadcast_action)
-> bot streamer.start/stop, guarded by isActive() so it's idempotent.
- Per-turn IPC uses a thread-local (reply/turn_state.py) instead of threading
params through the whole engine chain: bridge sets broadcasting in, tool
records the directive out; Tool.execute exposes broadcasting on ToolContext.
Bot typecheck clean; brain covered by tests/test_set_broadcast.py (+ existing
routing tests). Specs + docs updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
STREAM_BROWSER becomes the broadcast *capability* master flag; the live
screen-share state (new ToolContext.broadcasting, passed per turn by the bot)
decides the backend:
- master off -> broadcast disabled, always Gemini
- master on + live on -> on-screen Chrome (visible on the stream)
- master on + live off -> Gemini
context.broadcasting is None outside the voice path (evals, text entry, older
bot) and falls back to the master flag, so current behaviour is unchanged.
This is the brain-side foundation; bot-side wiring (bridge passes broadcast
state, auto-broadcast on voice join, voice on/off toggle) follows.
Specs + docs/llm_contexts.md updated. Covered by
tests/test_web_search_broadcast_routing.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
yolo auto-approves every tool call, so a real-time search query could in
principle trigger write/shell tools. Default approval mode still auto-runs the
CLI's read-only web search in headless mode but never silently approves
destructive tools. Verified end-to-end: a grounded query returns a current
answer in ~23s with the account OAuth login. Test asserts yolo is absent;
specs and docs/llm_contexts.md updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a GEMINI_AUTH=oauth (default) sub-mode that shells out to the Gemini CLI
using the user's Google-account login instead of an API key. gemini_cli_search()
runs `gemini -p <query> -o json --skip-trust --approval-mode yolo`, strips
GEMINI_API_KEY/GOOGLE_API_KEY and sets GOOGLE_GENAI_USE_GCA=true so the CLI
selects the account OAuth method and fails fast when no login exists. Bounded by
a 30s timeout and fail-open to the DDG/Brave/Wikipedia cascade on any failure
(CLI missing, login expired, quota 429, timeout). GEMINI_AUTH=apikey keeps the
legacy REST path. Specs and docs/llm_contexts.md updated; behaviour covered by
tests/test_realtime_gemini_cli.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
