4 Commits

Author SHA1 Message Date
javis-bot
f89246a14d docs(docker): clarify userbot mode in compose/run-bot, bot token optional 2026-06-12 21:26:00 +09:00
javis-bot
8a2a109d5e feat(brain): make OUTPUT_LANGUAGE lock robust on small models
Harden the reply-language lock so qwen2.5:3b reliably stays in the locked
language instead of leaking the query language back in:

- reply_language_directive(): single resolver with clear precedence —
  explicit OUTPUT_LANGUAGE lock wins over the Piper/Chatterbox English-only
  fallback (this deployment's actual TTS is Korean MeloTTS, so the legacy
  English lock was both wrong and contradicting the Korean lock).
- Stronger, override-explicit directive wording, inserted near the FRONT of
  the system prompt so a small model gives it primacy over the persona.
- build_system_prompt(output_language=...): rewrite the persona's "in the
  user's language" clause to the locked language so the persona stops
  fighting the lock.
- docs/llm_contexts.md: document the resolver, precedence, and placement.

Live-verified on the running brain (qwen2.5:3b): Korean voice-style input
and a cold English query both return fully Korean replies with no CJK/Hanja
leak. Tests cover unset/set/agnostic/whitespace + precedence + persona rewrite.
2026-06-12 21:18:47 +09:00
javis-bot
006a32276a feat(brain): add OUTPUT_LANGUAGE reply-language lock
Add an optional OUTPUT_LANGUAGE env var that forces every reply into a
single language. When set, output_language_directive() injects a "respond
only in <language>" instruction (also forbidding other scripts) into the
chat loop's system prompt, next to the existing TTS English-only lock.
Empty (default) keeps the multilingual "reply in the user's language"
behaviour, so upstream is unaffected.

For the Korean-only deployment this also suppresses the occasional trailing
CJK/Hanja fragment qwen2.5:3b leaks on free-form chit-chat.

- system_prompt.py: language-agnostic output_language_directive() helper
- engine.py: read OUTPUT_LANGUAGE, append directive in _build_initial_system_message
- docker-compose.yml + .env.example: document/pass the new var
- docs/llm_contexts.md: note the new gating on the main reply context
- tests: cover unset/set/agnostic/whitespace cases
2026-06-12 21:08:44 +09:00
javis-bot
40877b65b3 docs(env): mark DISCORD_BOT_TOKEN optional — blank runs userbot mode 2026-06-12 21:01:08 +09:00
7 changed files with 194 additions and 19 deletions

View File

@@ -6,6 +6,10 @@
# ---------------------------------------------------------------------------
# Discord bot (normal bot account) — voice I/O + slash commands
# ---------------------------------------------------------------------------
# OPTIONAL — leave BLANK to run in userbot/selfbot mode (a single user account
# does voice + broadcast; see DISCORD_SELFBOT_TOKEN below). When this is empty
# and a selfbot token is present, the app runs as a userbot automatically.
# Only fill this in if you specifically want the legacy normal-bot path.
# From https://discord.com/developers/applications → your app
DISCORD_BOT_TOKEN=
DISCORD_APP_ID=
@@ -55,6 +59,12 @@ OLLAMA_CHAT_MODEL=qwen2.5:3b
OLLAMA_EMBED_MODEL=nomic-embed-text
WHISPER_MODEL=small
# Lock every reply to one language, e.g. OUTPUT_LANGUAGE=Korean. Leave BLANK to
# keep the default behaviour of replying in whatever language the user wrote in.
# A fixed value also suppresses stray characters from other scripts (e.g. the
# occasional trailing CJK fragment small models leak on free-form chat).
OUTPUT_LANGUAGE=
# ---------------------------------------------------------------------------
# Docker desktop (VNC) — used only by the container image
# ---------------------------------------------------------------------------

View File

@@ -7,9 +7,11 @@
# Just bring it up — everything (incl. Ollama models) comes up automatically:
# docker compose up -d --build
#
# The Discord token can be added LAST: without it the desktop, brain bridge,
# Ollama and models all run; only the bot waits. Then put DISCORD_BOT_TOKEN in
# .env and re-run `docker compose up -d`.
# The Discord credential can be added LAST: without it the desktop, brain
# bridge, Ollama and models all run; only the bot waits. This deployment runs
# in userbot mode, so put DISCORD_SELFBOT_TOKEN in .env and re-run
# `docker compose up -d`. (A normal-bot DISCORD_BOT_TOKEN is optional and only
# needed for the legacy slash-command bot; leave it blank for userbot mode.)
#
# Watch the desktop: VNC viewer -> localhost:5901 (or browser -> localhost:6080)
# ============================================================================
@@ -65,6 +67,8 @@ services:
WHISPER_MODEL: ${WHISPER_MODEL:-small}
WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
# Optional single-language lock for replies (empty = user's own language).
OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-}
BRIDGE_URL: http://127.0.0.1:8765
depends_on:
- ollama

View File

@@ -1,9 +1,11 @@
#!/usr/bin/env bash
# Wait for the brain bridge, then run the Discord bot.
#
# The Discord token is intentionally deferred: if DISCORD_BOT_TOKEN is not set
# The Discord credential is intentionally deferred: if no usable token is set
# yet, the rest of the stack (desktop, bridge, ollama) still runs fully. The bot
# just waits. Add the token to .env and `docker compose up -d` to start it.
# just waits. Add a token to .env (DISCORD_SELFBOT_TOKEN for userbot mode, or
# DISCORD_BOT_TOKEN + DISCORD_APP_ID for the legacy normal bot) and
# `docker compose up -d` to start it.
set -e
cd /app/bot

View File

@@ -12,7 +12,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
- **Inputs**:
- Redacted user query
- Recent dialogue (last 5 minutes), including in-loop tool-call + tool-role messages from prior replies within the active conversation (tool carryover, `DialogueMemory.record_tool_turn` / `get_recent_turns_with_tools` in [src/jarvis/memory/conversation.py](src/jarvis/memory/conversation.py); per-prompt cap via `cfg.tool_carryover_max_turns` / `tool_carryover_per_entry_chars`; storage cap `_tool_turns_max_storage = 16`; cleared on `stop` signal AND on new-conversation entry; UNTRUSTED WEB EXTRACT fence markers preserved on truncation; both `content` and `tool_calls[*].function.arguments` scrubbed on write)
- Unified system prompt from [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) + ASR note + tool-protocol guidance
- Unified system prompt from [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) + ASR note + tool-protocol guidance. Reply language is resolved by `reply_language_directive(OUTPUT_LANGUAGE, cfg.tts_engine)`: an explicit `OUTPUT_LANGUAGE` env lock wins (forces "reply only in `<language>`", also forbidding other scripts so small models stop leaking trailing CJK/Hanja); else a Piper/Chatterbox TTS forces English (English-only voices); else (multilingual TTS, no lock) the assistant replies in the user's own language. The directive is inserted near the FRONT of the guidance list so a small model gives it primacy, and when the lock is set `build_system_prompt()` also rewrites the persona's "in the user's language" clause to the locked language so the persona does not contradict the lock. Gated in `_build_initial_system_message()` at [engine.py](src/jarvis/reply/engine.py).
- **Warm profile block** (query-agnostic User + Directives excerpt from the knowledge graph, composed by `build_warm_profile()` / `format_warm_profile_block()` in [src/jarvis/memory/graph_ops.py](src/jarvis/memory/graph_ops.py) at Step 3.5 of `reply()`; no LLM call, pure SQLite read; injected unconditionally so personalisation is the default; result cached in `DialogueMemory._hot_cache` under `DialogueMemory.WARM_PROFILE_CACHE_KEY` for the lifetime of the active conversation. Invalidated on `stop`, on new-conversation entry, AND on User/Directives graph mutations via the listener registered in [src/jarvis/daemon.py](src/jarvis/daemon.py) against `register_graph_mutation_listener` in [src/jarvis/memory/graph.py](src/jarvis/memory/graph.py); World-branch writes are ignored)
- Digested memory enrichment (optional, see #4)
- Time + location context (re-injected each turn)

View File

@@ -5,10 +5,11 @@ Handles memory enrichment, tool planning and execution.
"""
from __future__ import annotations
import os
from typing import Optional, TYPE_CHECKING
from ..utils.redact import redact
from ..system_prompt import build_system_prompt
from ..system_prompt import build_system_prompt, reply_language_directive
from ..tools.registry import run_tool_with_retries, generate_tools_description, generate_tools_json_schema, BUILTIN_TOOLS
from ..tools.builtin.stop import STOP_SIGNAL
from ..debug import debug_log
@@ -1424,7 +1425,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
action_plan = strip_memory_directives(action_plan)
_assistant_name = str(getattr(cfg, "wake_word", "jarvis") or "jarvis").strip().capitalize()
_persona_prompt = build_system_prompt(_assistant_name)
_persona_prompt = build_system_prompt(_assistant_name, os.environ.get("OUTPUT_LANGUAGE"))
def _build_initial_system_message() -> str:
guidance = [_persona_prompt.strip()]
@@ -1432,14 +1433,19 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# Add model-size-appropriate prompt components
guidance.extend(prompts.to_list())
# Both current TTS engines (Piper, Chatterbox) only support English.
# Responding in another language would produce garbled audio.
# Remove this constraint when a multilingual TTS engine is added.
tts_engine = getattr(cfg, 'tts_engine', 'piper')
if tts_engine in ('piper', 'chatterbox'):
guidance.append(
"Always respond in English regardless of the language the user speaks in."
)
# Reply-language policy: an explicit OUTPUT_LANGUAGE lock wins, else
# Piper/Chatterbox TTS forces English (English-only voices), else the
# assistant replies in the user's own language. See
# reply_language_directive() for the precedence rationale.
# Placed at the FRONT (after the persona header) so a small model gives
# it primacy over the persona's "use the user's language" lines — a tail
# instruction loses to those when the query itself is in another language.
_lang_directive = reply_language_directive(
os.environ.get("OUTPUT_LANGUAGE"),
getattr(cfg, "tts_engine", "piper"),
)
if _lang_directive:
guidance.insert(1, _lang_directive)
if warm_profile_block:
# Pre-query, query-agnostic user context. Lives OUTSIDE the

View File

@@ -6,6 +6,8 @@ who renames the wake word (e.g. "Friday") gets a butler with the matching
name rather than a persona hardcoded to "Jarvis".
"""
from typing import Optional
_SYSTEM_PROMPT_TEMPLATE: str = (
"Persona: you are a British butler named {name} — polite, composed, quietly amused, and "
"quietly enjoying yourself. Default voice is dry, witty, and lightly sarcastic: you notice "
@@ -79,11 +81,80 @@ _SYSTEM_PROMPT_TEMPLATE: str = (
)
def build_system_prompt(assistant_name: str = "Jarvis") -> str:
def build_system_prompt(
assistant_name: str = "Jarvis", output_language: Optional[str] = None
) -> str:
"""Render the persona prompt with the configured assistant name.
The name comes from the user's wake word (capitalised); defaults to
"Jarvis" when no config is available (tests, eval harnesses).
When ``output_language`` is set (a single-language deployment), the
persona's "reply in the user's language" clause is rewritten to that
language so the persona does not contradict the OUTPUT_LANGUAGE lock — a
small model otherwise honours the persona's instruction to mirror the
query language and leaks the other language back in.
"""
name = (assistant_name or "Jarvis").strip() or "Jarvis"
return _SYSTEM_PROMPT_TEMPLATE.format(name=name)
prompt = _SYSTEM_PROMPT_TEMPLATE.format(name=name)
lang = (output_language or "").strip()
if lang:
prompt = prompt.replace("in the user's language", f"in {lang}")
return prompt
def output_language_directive(language: Optional[str]) -> Optional[str]:
"""Return a 'respond only in <language>' instruction, or None when unset.
Deployments that serve a single language set ``OUTPUT_LANGUAGE`` (read by
the reply engine). When it is empty/None the assistant keeps its default
multilingual behaviour of replying in whatever language the user wrote in,
so this returns ``None`` and no directive is injected.
The instruction is language-agnostic — it names whatever language string it
is given — and forbids mixing in other scripts. That exclusivity also
suppresses the occasional trailing CJK/Hanja fragment some small models
leak on free-form chit-chat.
"""
lang = (language or "").strip()
if not lang:
return None
return (
f"CRITICAL OUTPUT RULE: write your ENTIRE reply only in {lang}. Even if "
f"the user writes in English or any other language, you must still reply "
f"only in {lang}. This rule overrides every other instruction about "
f"matching or using the user's language. Never mix in words, characters, "
f"or punctuation from any other language or script."
)
# TTS engines that can only synthesise English. Replying in another language
# with one of these produces garbled audio, so those deployments force English.
_TTS_ENGLISH_ONLY = frozenset({"piper", "chatterbox"})
# Kept verbatim for backward compatibility with anything asserting on the wording.
ENGLISH_ONLY_DIRECTIVE = (
"Always respond in English regardless of the language the user speaks in."
)
def reply_language_directive(
output_language: Optional[str], tts_engine: Optional[str]
) -> Optional[str]:
"""Resolve the reply-language instruction for the chat loop, or None.
Precedence:
1. An explicit ``output_language`` lock wins — the deployment serves a
single language and owns a TTS voice that can speak it (e.g. Korean
MeloTTS). This intentionally overrides the English-only fallback.
2. Otherwise, a Piper/Chatterbox TTS engine can only synthesise English,
so force English to avoid garbled audio.
3. Otherwise (multilingual TTS, no lock) → None: the assistant replies in
the user's own language.
"""
forced = output_language_directive(output_language)
if forced:
return forced
if (tts_engine or "piper").strip().lower() in _TTS_ENGLISH_ONLY:
return ENGLISH_ONLY_DIRECTIVE
return None

View File

@@ -5,7 +5,12 @@ wake word to e.g. "Friday" produces a butler named Friday, not one still
hardcoded to Jarvis.
"""
from jarvis.system_prompt import build_system_prompt
from jarvis.system_prompt import (
build_system_prompt,
output_language_directive,
reply_language_directive,
ENGLISH_ONLY_DIRECTIVE,
)
class TestBuildSystemPrompt:
@@ -26,3 +31,80 @@ class TestBuildSystemPrompt:
assert "named Jarvis" in build_system_prompt("")
assert "named Jarvis" in build_system_prompt(" ")
assert "named Jarvis" in build_system_prompt(None) # type: ignore[arg-type]
def test_default_keeps_user_language_clause(self):
# Without a lock, the persona still mirrors the user's language.
assert "in the user's language" in build_system_prompt("Jarvis")
def test_language_lock_rewrites_user_language_clause(self):
# With a lock, the contradicting "user's language" clause is rewritten
# so the persona does not fight the OUTPUT_LANGUAGE directive.
prompt = build_system_prompt("Jarvis", "Korean")
assert "in the user's language" not in prompt
assert "in Korean" in prompt
class TestOutputLanguageDirective:
"""A deployment may lock replies to a single language via OUTPUT_LANGUAGE.
Unset (the default) must keep the assistant's multilingual behaviour of
replying in the user's own language, so the helper returns None and no
directive is injected.
"""
def test_unset_returns_none(self):
assert output_language_directive(None) is None
assert output_language_directive("") is None
assert output_language_directive(" ") is None
def test_set_language_is_named_and_exclusive(self):
directive = output_language_directive("Korean")
assert directive is not None
assert "Korean" in directive
# Must force exclusivity, not merely prefer the language.
assert "only" in directive.lower()
def test_language_agnostic(self):
# The helper takes any language string — no hardcoded single language.
assert "French" in (output_language_directive("French") or "")
assert "日本語" in (output_language_directive("日本語") or "")
def test_strips_surrounding_whitespace(self):
directive = output_language_directive(" Korean ")
assert directive is not None
assert "Korean" in directive
assert " Korean" not in directive
class TestReplyLanguageDirective:
"""Precedence: explicit OUTPUT_LANGUAGE lock > English-only TTS > free.
The lock must override the Piper/Chatterbox English fallback, because a
deployment that sets OUTPUT_LANGUAGE (e.g. Korean) also runs a TTS voice
that can speak it. Without this, the English lock and the Korean lock
contradict each other and the model reverts to English.
"""
def test_lock_overrides_english_only_tts(self):
directive = reply_language_directive("Korean", "piper")
assert directive is not None
assert "Korean" in directive
assert directive != ENGLISH_ONLY_DIRECTIVE
def test_english_only_tts_forces_english_without_lock(self):
assert reply_language_directive(None, "piper") == ENGLISH_ONLY_DIRECTIVE
assert reply_language_directive("", "chatterbox") == ENGLISH_ONLY_DIRECTIVE
def test_no_lock_multilingual_tts_is_free(self):
# A non-English-only engine (e.g. melo) with no lock → reply in the
# user's own language, so no directive.
assert reply_language_directive(None, "melo") is None
def test_unknown_tts_defaults_to_english_only(self):
# Preserves the original getattr(cfg, 'tts_engine', 'piper') default:
# an unknown/missing engine is treated conservatively as English-only.
assert reply_language_directive(None, None) == ENGLISH_ONLY_DIRECTIVE
def test_lock_wins_even_with_multilingual_tts(self):
directive = reply_language_directive("Korean", "melo")
assert directive is not None and "Korean" in directive