Compare commits

..

7 Commits

Author SHA1 Message Date
javis-bot
4993942040 chore(app): version 0.1.22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 10:11:59 +09:00
javis-bot
ff78c0ac41 feat(app): the LLM always takes one request at a time, and only the latest call is answered by default
A chat channel answering every message no longer starts Strata with
--parallel 2: on the 16 GB card that read prompts about three times
slower (2026-11-16, answers took 10-20 s). Chat and voice answers now
take turns at Strata. response.policy defaults to latest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 10:01:37 +09:00
javis-bot
8c61d7a821 perf(llm): short one-shot calls say so to Strata
Every call that no later request continues (call_llm_direct,
call_llm_streaming, the warm-up and the v1 intent judge) now sends
"strata_checkpoint": false on the OpenAI-compatible path. Strata's
Qwen3.6 engine then gives that call's session away first, so with two or
more requests at once a reply's tool turn keeps its session through the
short calls made between its steps. The reply loop's chat_with_messages
is not marked (its next step continues it); the Ollama API gets no new
field. With one request at once nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 07:06:25 +09:00
javis-bot
07615ee7a1 chore(app): version 0.1.21
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 21:13:39 +09:00
javis-bot
4d0ae91e8a perf(reply): v2 tool turns get the tool guidance in short, and a shorter reply-language default
Strata reads every reply prompt from the start in 2,048-token steps. The v2
compact prompt carried v1's full tool guidance and constraints (about 900
Qwen3.6 tokens) on every turn with a data tool, and the reply-language
default (about 120 tokens) twice.

COMPACT_TOOL_GUIDANCE keeps the same rules (act instead of refusing, facts
only from the result, look up named things, auto-derived and self-contained
arguments) in about a quarter of the tokens, without English examples; the
reply-language default says the same in half. v1's prompts are unchanged.

Measured on Strata (Qwen3.6, 20 lines of recent conversation): a weather
turn's prompt 3,436 -> 2,722 tokens, a search turn's second call
4,096 -> 3,383. Evals (compact prompt + intent judge, Qwen3.6): 24/24
before, 30/30 twice after (weather location, follow-up arguments, tool-skip
confabulation, search language, reply language, Possessor fidelity, reply
length, multi-turn context).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 21:13:08 +09:00
javis-bot
7da5bca783 fix(reply): a one-step turn whose model skips the tool twice is planned and guarded
After the one nudge, a second answer without the tool no longer goes out
as it is: the turn takes the planned path (the planner runs then, one
extra call only on this failure) and the skipped-tool guard runs the
planned tool, so a made-up "24 degrees" answer cannot reach the user. A
plan that needs no tool (a question back to the user) still lets the
answer go out.

Evals (qwen3:8b, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1, 3 runs):
the five tool-skip/weather/screen/follow-up/router files 72/84 (71/84 on
0.1.20, same known failures); Possessor director-then-filmography 0/3,
also 0/3 on 0.1.20 (qwen3:8b stops after one search; webSearch turns are
still planned).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:09:11 +09:00
javis-bot
a6932d9b16 perf(reply): v2 turns with one one-step tool skip the planner
When the router's only pick is a tool that does its job in one call
(getWeather, logMeal, fetchMeals, screenView) and the chat model calls
tools natively, the planner no longer runs: on Strata it averaged 2.5 s
per tool turn (2026-11-16 16:45-16:58 engine log, 9 calls) before the
answer could start. Search, browsers, schedule, notes, chat channels and
picks of two or more tools still plan. The rule reads the router's pick,
not the words, so it is language-agnostic.

Without a plan the skipped-tool guard has nothing to run, so these turns
get one nudge instead: an answer given before any tool ran is dropped and
the model is asked to call the tool (or ask for a missing detail).

Evals (qwen3:8b, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1, 3 runs of
tool_skip_confabulation, weather_autoderive_location, screen_view_tool,
followup_supplies_missing_tool_arg, tool_router_implicit): 71/84 before,
72/84 after, same known failures; planner calls 14 -> 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:00:36 +09:00
24 changed files with 390 additions and 83 deletions

2
app/Cargo.lock generated
View File

@@ -1520,7 +1520,7 @@ checksum = "3bd1574e617a9a6c1f8f8181bd7e841e1f20e4277607ef28945765188d6284d4"
[[package]]
name = "jarvis-app"
version = "0.1.20"
version = "0.1.22"
dependencies = [
"async-trait",
"base64 0.22.1",

View File

@@ -1,6 +1,6 @@
[package]
name = "jarvis-app"
version = "0.1.20"
version = "0.1.22"
edition = "2021"
rust-version = "1.85"
description = "Jarvis v2 Windows app core: Discord voice, voice worker supervisor, chat channels"

4
app/shell/Cargo.lock generated
View File

@@ -2637,7 +2637,7 @@ checksum = "3bd1574e617a9a6c1f8f8181bd7e841e1f20e4277607ef28945765188d6284d4"
[[package]]
name = "jarvis-app"
version = "0.1.20"
version = "0.1.22"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -2666,7 +2666,7 @@ dependencies = [
[[package]]
name = "jarvis-shell"
version = "0.1.20"
version = "0.1.22"
dependencies = [
"jarvis-app",
"serde_json",

View File

@@ -1,6 +1,6 @@
[package]
name = "jarvis-shell"
version = "0.1.20"
version = "0.1.22"
edition = "2021"
rust-version = "1.85"
description = "Jarvis v2 management app window (Tauri 2 over the app core's AdminApi)"

View File

@@ -624,9 +624,9 @@ start it first, so nothing answers before the brain can, and stop it last.
environment, `runtime/strata` as the working folder, one process group, lines to the `llm` log
tab). It downloads the model on the first start, starts the engine and returns; the app then
waits (up to 30 minutes) until `/v1/models` lists the model and takes its id. `--restart`
replaces a server left with other choices; the same choices are reused. `--parallel` is 1 (faster
on a 16 GB card, M0) and 2 when a chat channel answers every message, so its answers do not wait
behind voice answers. Pictures (`--vision`) are off unless screen viewing is available ("Screen viewing"): then `gpu`
replaces a server left with other choices; the same choices are reused. `--parallel` is always 1 (faster
on a 16 GB card, M0; with 2 the answers were up to 3 times slower), also when a chat channel
answers every message: its answers take turns with the voice answers. Pictures (`--vision`) are off unless screen viewing is available ("Screen viewing"): then `gpu`
in the normal stage and `cpu` in the game stage (M0). Stopping the services
runs `strata launch --stop` (ended after 2 minutes, with everything it started) and then checks
that the port refuses connections; so does a failed or cancelled start, so nothing half-started

View File

@@ -89,7 +89,7 @@ Settings (section `response`):
| Key | Default | Meaning |
|---|---|---|
| `response.policy` | `"first"` | `first` (처음 대화만), `inOrder` (들어온 순서대로), `latest` (마지막 대화만), `important` (제일 중요한 대화만). |
| `response.policy` | `"latest"` | `first` (처음 대화만), `inOrder` (들어온 순서대로), `latest` (마지막 대화만), `important` (제일 중요한 대화만). |
| `response.queueMax` | `5` | `inOrder`: most waiting requests; past it the oldest is dropped. 1-20. |
| `response.collectMs` | `700` | `important`: how long candidates are collected before choosing. 0-3000. |
| `response.settleMs` | `800` | Longest wait for an earlier-ended utterance that is still being transcribed. 0-3000. |
@@ -167,10 +167,9 @@ never takes the voice turn and never goes through the response policy.
(`reply_delivered`).
- The worker writes voice and chat answers in separate lanes (`voice_worker.spec.md`), so a voice
answer never waits behind another person's chat answer; one person's answers are written one at a
time, in the order they asked. For the two to run at the same time the LLM must take two
requests at once (`docs/v2/conversation.md`, "LLM 동시 요청 수"): until the app starts Strata itself
(M5), Strata is started with `--parallel 2` when some channel answers everything; with
`--parallel 1` the two lanes' LLM calls take turns at Strata.
time, in the order they asked. The LLM takes one request at a time (`docs/v2/conversation.md`,
"LLM 동시 요청 수"; `--parallel 1`, also when a channel answers everything), so the two lanes' LLM
calls take turns at Strata.
## The conversation loop

View File

@@ -36,7 +36,7 @@ pub struct PolicySettings {
impl Default for PolicySettings {
fn default() -> Self {
Self {
kind: PolicyKind::First,
kind: PolicyKind::Latest,
queue_max: 5,
collect: Duration::from_millis(700),
settle: Duration::from_millis(800),

View File

@@ -447,7 +447,7 @@ fn parse_call(raw: &Map<String, Value>, wake_words: &[String], d: &CallSettings)
fn parse_response(raw: &Map<String, Value>, d: &PolicySettings) -> Result<PolicySettings, SettingsError> {
let r = section(raw, "response")?;
let kind = match get_str(r, "policy", "response", "first")?.as_str() {
let kind = match get_str(r, "policy", "response", "latest")?.as_str() {
"first" => PolicyKind::First,
"inOrder" => PolicyKind::InOrder,
"latest" => PolicyKind::Latest,
@@ -914,6 +914,11 @@ mod tests {
assert_eq!(load_settings(&dir.join("nope.json")).unwrap(), Settings::default());
}
#[test]
fn by_default_only_the_latest_call_is_answered() {
assert_eq!(parse_settings(&json!({})).unwrap().response.kind, PolicyKind::Latest);
}
#[test]
fn reads_a_full_file_with_bom() {
let dir = std::env::temp_dir().join(format!("jarvis-settings-full-{}", std::process::id()));

View File

@@ -94,7 +94,7 @@ pub fn fields() -> Vec<Field> {
f("call.followUpMs", Channels, Int { min: 0, max: 5000 }, json!(1500), "호출어 뒤 이어 말하기 대기(ms)"),
f("call.earcon", Channels, Bool, json!(true), "부르면 효과음"),
f("response.policy", Channels, Choice { values: &["first", "inOrder", "latest", "important"] },
json!("first"), "동시 대화 응답 정책"),
json!("latest"), "동시 대화 응답 정책"),
f("response.queueMax", Channels, Int { min: 1, max: 20 }, json!(5), "기다리는 요청 최대 개수"),
f("response.collectMs", Channels, Int { min: 0, max: 3000 }, json!(700), "중요도 비교 대기(ms)"),
f("response.settleMs", Channels, Int { min: 0, max: 3000 }, json!(800), "먼저 끝난 말 전사 대기(ms)"),

View File

@@ -41,12 +41,6 @@ pub struct LlmStatus {
pub note: Option<String>,
}
/// Requests answered at once: one is faster on a 16 GB card (M0), two only when a chat channel
/// answers every message (so its answers do not wait behind the voice answers).
pub fn parallel(settings: &Settings) -> u32 {
if settings.chat_channels.iter().any(|c| c.answer_all) { 2 } else { 1 }
}
/// `strata launch` for the services: the M0 kit's normal placement, models in `models/llm`, this
/// computer only. No pictures yet (nothing in v2 sends one; reading them takes graphics memory).
/// `--restart` replaces a server started with other choices; the same choices are reused.
@@ -68,8 +62,9 @@ pub fn launch_args(layout: &Layout, settings: &Settings, conf: &Conf, sets: &[St
a
}
/// The context and requests at once of a stage: the settings' in the normal stage; in a game 16K
/// (or less when the settings ask for less) and one request at a time.
/// The context and requests at once of a stage: the settings' context in the normal stage, in a
/// game 16K (or less when the settings ask for less); always one request at a time (faster on a
/// 16 GB card, M0; chat answers take turns with the voice answers).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Conf {
pub context: u32,
@@ -82,7 +77,7 @@ pub fn conf(settings: &Settings, stage: Stage) -> Conf {
let vision = vision(stage, crate::screen::available(settings));
match stage {
Stage::Gaming => Conf { context: crate::placement::GAME_CONTEXT.min(settings.llm.context), parallel: 1, vision },
_ => Conf { context: settings.llm.context, parallel: parallel(settings), vision },
_ => Conf { context: settings.llm.context, parallel: 1, vision },
}
}
@@ -687,12 +682,12 @@ pub(crate) mod tests {
}
#[test]
fn a_chat_channel_answering_everything_gets_a_second_request_slot() {
fn the_llm_takes_one_request_at_a_time_also_with_a_chat_channel_answering_everything() {
let mut s = Settings::default();
assert_eq!(parallel(&s), 1);
s.chat_channels.push(ChatChannelSettings { id: 1, name: String::new(), description: String::new(), read: true,
send: true, edit: true, react: true, delete: false, pin: false, answer_all: true, sync_discord: true });
assert_eq!(parallel(&s), 2);
let a = launch_args(&layout("llm-one"), &s, &conf(&s, Stage::Normal), &[]);
assert_eq!(a[a.iter().position(|x| x == "--parallel").unwrap() + 1], "1");
}
#[test]
@@ -975,7 +970,7 @@ pub(crate) mod tests {
#[cfg(unix)]
#[tokio::test]
async fn the_measurement_is_kept_per_requests_at_once_and_per_card() {
async fn the_measurement_is_kept_per_context_and_per_card() {
let l = layout("llm-place-keys");
let llm = Llm::with_exe(l.clone(), fake_placed(&l)).with_gpu(card(&l, "GPU-a", 0));
let port = server_after_launch(&l, "list").await;
@@ -983,13 +978,11 @@ pub(crate) mod tests {
llm.start(&one).await.unwrap();
assert_eq!(launches(&l).len(), 2, "measured once");
llm.stop().await.unwrap();
// A chat channel that answers everything: two requests at once, measured again.
// Another context: measured again.
std::fs::remove_file(l.data.join("calls")).unwrap();
let two = parse_settings(&json!({"llm": {"port": port}, "chatChannels": [{"id": "123456789012345678", "answer": "everything"}]})).unwrap();
assert_eq!(parallel(&two), 2);
llm.start(&two).await.unwrap();
assert_eq!(launches(&l).len(), 2, "measured for two at once");
assert!(launches(&l)[0].contains("--parallel 2"));
let short = parse_settings(&json!({"llm": {"port": port, "context": 16384}})).unwrap();
llm.start(&short).await.unwrap();
assert_eq!(launches(&l).len(), 2, "measured for the other context");
llm.stop().await.unwrap();
// Another card (same size): measured again.
std::fs::remove_file(l.data.join("calls")).unwrap();
@@ -997,7 +990,7 @@ pub(crate) mod tests {
llm.start(&one).await.unwrap();
assert_eq!(launches(&l).len(), 2, "measured for the new card");
let probe: Value = serde_json::from_str(&std::fs::read_to_string(l.data.join("config").join("llm-probe.json")).unwrap()).unwrap();
assert_eq!((probe["32768/1"]["gpu"].clone(), probe["32768/2"]["gpu"].clone()), (json!("GPU-b"), json!("GPU-a")));
assert_eq!((probe["32768/1"]["gpu"].clone(), probe["16384/1"]["gpu"].clone()), (json!("GPU-b"), json!("GPU-a")));
llm.stop().await.unwrap();
}

View File

@@ -2,7 +2,7 @@
Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it is gated. This is the reference for optimising the app's main bottleneck (LLM latency). Keep it in sync with the code — see the note at the bottom.
**Transport (all contexts below).** `ollama_base_url` selects the API. A URL that ends in `/v1` (v2: Strata at `http://127.0.0.1:8080/v1`, which the app starts and writes with the model id Strata lists into the brain's `config/brain.json`, `app/src/strata.rs`; also llama.cpp or Ollama's own `/v1`) uses the OpenAI-compatible API: chat goes to `/chat/completions`, embeddings to `/embeddings`, and the warm-up is a one-token chat completion. Any other URL uses Ollama's `/api/chat`, `/api/generate` and `/api/embeddings`. The conversion lives in `src/jarvis/llm.py` only: callers keep sending Ollama-shaped messages and reading Ollama-shaped responses (`message.content`, `message.tool_calls` with dict arguments, `message.thinking` from `reasoning_content`). On the OpenAI path `num_ctx` and `keep_alive` are not sent (the server owns its context size and residency), `num_predict` becomes `max_tokens`, and the thinking switch becomes `chat_template_kwargs.enable_thinking`. The `num_ctx: 8192` limits listed below apply to the Ollama path. Ollama's own `/v1` ignores `num_ctx` and runs at its default context (2,048 tokens here), which cuts the system prompt; to test the OpenAI path on Ollama use a model tag created with a larger `num_ctx`. A server with no embedding model (Strata) answers `/embeddings` with an error and every embedding consumer fails open to its text path. In v2 the app also sets Strata's context per performance stage (`app/src/admin.spec.md`, "Performance stages"): the settings' `llm.context` (default 32,768 tokens) normally and 16,384 tokens with one request at a time while a game runs; while it switches the voice worker starts no LLM call (requests wait, at most 180 s, `voice_worker.spec.md` "LLM switches"), and while paused (a game with no room for the model) every brain call fails at once.
**Transport (all contexts below).** `ollama_base_url` selects the API. A URL that ends in `/v1` (v2: Strata at `http://127.0.0.1:8080/v1`, which the app starts and writes with the model id Strata lists into the brain's `config/brain.json`, `app/src/strata.rs`; also llama.cpp or Ollama's own `/v1`) uses the OpenAI-compatible API: chat goes to `/chat/completions`, embeddings to `/embeddings`, and the warm-up is a one-token chat completion. Any other URL uses Ollama's `/api/chat`, `/api/generate` and `/api/embeddings`. The conversion lives in `src/jarvis/llm.py` only: callers keep sending Ollama-shaped messages and reading Ollama-shaped responses (`message.content`, `message.tool_calls` with dict arguments, `message.thinking` from `reasoning_content`). On the OpenAI path `num_ctx` and `keep_alive` are not sent (the server owns its context size and residency), `num_predict` becomes `max_tokens`, and the thinking switch becomes `chat_template_kwargs.enable_thinking`. Every call that no later request continues (all `call_llm_direct` and `call_llm_streaming` calls, the warm-up and the v1 intent judge; not the reply loop's `chat_with_messages`, whose next step continues it) carries `"strata_checkpoint": false`: Strata's Qwen3.6 engine then gives that call's session away first, so with two or more requests at once a reply's tool turn keeps its session through the short calls made between its steps (with one request at once nothing changes, and Strata releases up to 0.1.59 ignore the field on this engine); other servers ignore it. The `num_ctx: 8192` limits listed below apply to the Ollama path. Ollama's own `/v1` ignores `num_ctx` and runs at its default context (2,048 tokens here), which cuts the system prompt; to test the OpenAI path on Ollama use a model tag created with a larger `num_ctx`. A server with no embedding model (Strata) answers `/embeddings` with an error and every embedding consumer fails open to its text path. In v2 the app also sets Strata's context per performance stage (`app/src/admin.spec.md`, "Performance stages"): the settings' `llm.context` (default 32,768 tokens) normally and 16,384 tokens with one request at a time while a game runs; while it switches the voice worker starts no LLM call (requests wait, at most 180 s, `voice_worker.spec.md` "LLM switches"), and while paused (a game with no room for the model) every brain call fails at once.
---
@@ -15,7 +15,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
- Redacted user query
- Recent dialogue (last 5 minutes), including in-loop tool-call + tool-role messages from prior replies within the active conversation (tool carryover, `DialogueMemory.record_tool_turn` / `get_recent_turns_with_tools` in [src/jarvis/memory/conversation.py](src/jarvis/memory/conversation.py); per-prompt cap via `cfg.tool_carryover_max_turns` / `tool_carryover_per_entry_chars`; storage cap `_tool_turns_max_storage = 16`; cleared on `stop` signal AND on new-conversation entry; UNTRUSTED WEB EXTRACT fence markers preserved on truncation; both `content` and `tool_calls[*].function.arguments` scrubbed on write)
- Unified system prompt from [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) + ASR note + tool-protocol guidance. Reply language is resolved by `reply_language_directive(lang, cfg.tts_engine)` where `lang = _resolve_output_language()` — the single source of truth that prefers the settings-web UI value (config JSON `output_language`) over the compose `OUTPUT_LANGUAGE` env, so changing the language in the settings page takes effect. An explicit lock wins (forces "reply only in `<language>`", also forbidding other scripts so small models stop leaking trailing CJK/Hanja); else a Piper/Chatterbox TTS forces English (English-only voices); else (multilingual TTS, no lock) the assistant replies in the user's own language. The v2 voice worker's brain (`src/voice_worker/brain.py`) runs with `tts_engine="melo"`, so v1's English-only Piper default never locks v2 voice or chat answers to English, and passes `default_language` (worker `--reply-language`, default `Korean`): with no lock, `default_language_directive()` makes replies come in that language unless the user explicitly asks for another one, and `build_system_prompt()` rewrites the persona's "in the user's language" clause to "in the reply language" (eval `evals/test_reply_language_default.py`). The directive is inserted near the FRONT of the guidance list so a small model gives it primacy, and the SAME resolved `lang` feeds `build_system_prompt()`, which rewrites the persona's "in the user's language" clause to the locked language so the persona cannot contradict the directive (previously the persona read the raw env while the directive read the config value, so a settings-UI change was honoured by one and ignored by the other). Gated in `_build_initial_system_message()` at [engine.py](src/jarvis/reply/engine.py).
- **v2 compact prompt** (`compact_prompt=True`, the v2 brain): `build_voice_persona()` replaces the v1 persona (about 1.3k characters instead of 7.4k: no single-user long-term memory or on-screen browser rules, answers as short as possible), and of the size-specific components only the ASR note stays; the full tool guidance (`tool_guidance` + `tool_constraints`) is added only when a tool other than `stop`, `toolSearchTool` and `chatChannels` is allowed, else the one-line `COMPACT_ACTION_RULE`. Measured on Strata (Qwen3.6, 2026-11-12): a greeting's reply prompt went from about 4,500 to 1,350 tokens and its prompt read from 1.07 s to 0.36 s (Strata reads in 2,048-token steps). The v2 brain also turns `stream_browser` off. The `[Context: ...]` time carries the 12-hour form beside the 24-hour one (`10:37 (10:37 AM)`): with the short prompt Qwen3.6 read 10:37 as 오후 (pm) 4 of 4 times.
- **v2 compact prompt** (`compact_prompt=True`, the v2 brain): `build_voice_persona()` replaces the v1 persona (about 1.3k characters instead of 7.4k: no single-user long-term memory or on-screen browser rules, answers as short as possible), and of the size-specific components only the ASR note stays; the tool guidance is added only when a tool other than `stop`, `toolSearchTool` and `chatChannels` is allowed, as `COMPACT_TOOL_GUIDANCE` (v1's `tool_guidance` + `tool_constraints` rules in about a quarter of the tokens), else the one-line `COMPACT_ACTION_RULE`; the reply-language default (`default_language_directive`, at the front and the end) is short too. Measured on Strata (Qwen3.6, 2026-11-16, four lines of recent conversation): a weather turn's reply prompt went from 3,075 to 2,360 tokens and a time question's from 2,115 to 2,031, inside one 2,048-token step (Strata reads about 0.44 s up to 2,048 tokens, 0.86 s up to 4,096, 1.25 s up to 6,144). Measured on Strata (Qwen3.6, 2026-11-12): a greeting's reply prompt went from about 4,500 to 1,350 tokens and its prompt read from 1.07 s to 0.36 s (Strata reads in 2,048-token steps). The v2 brain also turns `stream_browser` off. The `[Context: ...]` time carries the 12-hour form beside the 24-hour one (`10:37 (10:37 AM)`): with the short prompt Qwen3.6 read 10:37 as 오후 (pm) 4 of 4 times.
- **Tool honesty rules**: `TOOL_HONESTY_RULES` inside the persona, plus `tool_honesty_reminder(allowed_tools)` appended after every context block (just before the closing reply-language lock) naming the turn's tools. Added after M0 (2026-11-10) caught models answering past an available tool.
- **Operator instructions** (two sources, both framed "Additional instructions from the operator:" and appended near the end of the guidance list): the settings-UI `llm_instructions` config field, and every `*.md` file in `AGENTS_DIR` (default `/app/agents`, bind-mounted from `./agents`). The file-based set is read once per turn by `load_agent_instructions()` in [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) and concatenated in filename order, so dropping/editing a `.md` applies on the next reply with no rebuild/restart; fail-open to `""` when the folder is absent/empty/unreadable.
- **Warm profile block** (query-agnostic User + Directives excerpt from the knowledge graph, composed by `build_warm_profile()` / `format_warm_profile_block()` in [src/jarvis/memory/graph_ops.py](src/jarvis/memory/graph_ops.py) at Step 3.5 of `reply()`; no LLM call, pure SQLite read; injected unconditionally so personalisation is the default; result cached in `DialogueMemory._hot_cache` under `DialogueMemory.WARM_PROFILE_CACHE_KEY` for the lifetime of the active conversation. Invalidated on `stop`, on new-conversation entry, AND on User/Directives graph mutations via the listener registered in [src/jarvis/daemon.py](src/jarvis/daemon.py) against `register_graph_mutation_listener` in [src/jarvis/memory/graph.py](src/jarvis/memory/graph.py); World-branch writes are ignored)
@@ -161,7 +161,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
## 12. Task-list Planner (pre-flight decomposition, gates the whole turn)
- **File**: [src/jarvis/reply/planner.py](src/jarvis/reply/planner.py) — `plan_query()`.
- **Trigger**: once per reply, **after the tool router and before memory search**. **v2 brain** (`intent_judge=True`): runs only on turns where the router picked a tool (not for the sentinels `stop` or `toolSearchTool` alone; a router-picked `chatChannels` counts), so a turn without tools has one LLM call before its answer. Skipped when `cfg.planner_enabled = False`, when the query is shorter than `MIN_QUERY_CHARS` (4), or when no model / base URL is available.
- **Trigger**: once per reply, **after the tool router and before memory search**. **v2 brain** (`intent_judge=True`): runs only on turns where the router picked a tool (not for the sentinels `stop` or `toolSearchTool` alone; a router-picked `chatChannels` counts), so a turn without tools has one LLM call before its answer. Also skipped when the router's only pick is a one-step tool (`_ONE_STEP_TOOLS` in `engine.py`: `getWeather`, `logMeal`, `fetchMeals`, `screenView`) and the chat model calls tools natively; the chat model then gets the tool directly, and an answer it gives without the tool is dropped and followed by one nudge to call it; a second answer without the tool runs the planner after all (one extra call, only then) and the skipped-tool guard applies (planner.spec.md, "Intent judge (v2)"). Skipped when `cfg.planner_enabled = False`, when the query is shorter than `MIN_QUERY_CHARS` (4), or when no model / base URL is available.
- **Model / gating**: resolution chain `planner_model (override) → ollama_chat_model`. The planner tracks the chat model so upgrading the chat model (via setup wizard or config) automatically upgrades plan quality.
- **Inputs**: user query, dialogue context, the live time/location line (`LIVE CONTEXT:`, so a location-dependent ask plans `searchMemory topic='where the user lives'` only when the location is unknown or disabled), **router-narrowed** tool catalogue (names + one-line descriptions + argument names, built by `planner_tool_catalog()`) — not the full 30+ list. When the carry-over guard from #7 fires, the previous turn's failed tool name is unioned into this catalogue before the planner sees it, so the planner can plan a re-call without `toolSearchTool` round-tripping. **No** memory context — the planner decides *whether* memory is needed.
- **System prompt**: `_PROMPT_TEMPLATE` in `planner.py`. Teaches the `searchMemory topic='...'` directive for prior-conversation lookups, short imperative tool steps, angle-bracket entity placeholders, final synthesis step, same-language output (the words of search and play arguments too, repeated in the user message's closing line), no numbering.
@@ -215,7 +215,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
| 10 | Graph extraction | ~1/session | No (background) | LARGE |
| 11 | Graph best-child | 0-N | No (background) | SMALL (via router chain) |
| 11b | Graph node merge | 0-N (per node, batched) | No (background) | SMALL (via router chain) |
| 12 | Planner (plan_query) | 1 | yes (planner_enabled) | LARGE/SMALL (tracks chat model) |
| 12 | Planner (plan_query) | 0-1 (v2: tool turns except one-step tools, which plan only when the model skips the tool twice) | yes (planner_enabled) | LARGE/SMALL (tracks chat model) |
| 13 | Plan step resolver | 0-N (SMALL only) | auto by size + plan | SMALL (via router chain) |
| 14 | Tool-specific | per-tool | n/a | LARGE |
| 15 | Voice call judgements | 0 (per utterance / tie, outside the reply) | call mode / policy | LARGE (chat model) |

View File

@@ -40,12 +40,12 @@
| 정책 | 새 요청 처리 | 봇이 말하는 중에 새 요청이 오면 |
|---|---|---|
| **처음 대화만 (기본값)** | 봇이 답을 준비하거나 말하는 동안 들어온 새 요청은 버립니다. 지금 답이 끝나야 다음 요청을 받습니다. 여러 명이 거의 동시에 부르면 **말이 먼저 끝난 사람** 하나에 답합니다. 먼저 끝난 말이 아직 전사 중이면 호출인지 확인될 때까지 잠깐 기다립니다. | 지금 답을 끝까지 재생하고, 새 요청은 버립니다. |
| **처음 대화만** | 봇이 답을 준비하거나 말하는 동안 들어온 새 요청은 버립니다. 지금 답이 끝나야 다음 요청을 받습니다. 여러 명이 거의 동시에 부르면 **말이 먼저 끝난 사람** 하나에 답합니다. 먼저 끝난 말이 아직 전사 중이면 호출인지 확인될 때까지 잠깐 기다립니다. | 지금 답을 끝까지 재생하고, 새 요청은 버립니다. |
| 들어온 순서대로 | 요청마다 **말이 끝난 시각**으로 순번을 붙여 그 순서대로 답합니다. 전사가 끝난 순서가 아니므로 길게 말한 사람이 뒤로 밀리지 않습니다. 대기열 최대 길이를 넘으면 가장 오래된 요청부터 버립니다. | 지금 답을 끝까지 재생하고, 새 요청은 대기열에 넣습니다. |
| 마지막 대화만 | 대기열에 가장 최근 요청 하나만 남깁니다. | 지금 답을 멈추고 가장 최근 요청에 바로 답합니다. 멈춘 답은 다시 재생하지 않습니다. |
| **마지막 대화만 (기본값)** | 대기열에 가장 최근 요청 하나만 남깁니다. | 지금 답을 멈추고 가장 최근 요청에 바로 답합니다. 멈춘 답은 다시 재생하지 않습니다. |
| 제일 중요한 대화만 | 짧은 수집 시간(기본 0.7초)과 봇이 말하는 동안 들어온 요청을 모아 점수를 매기고, 가장 높은 하나에만 답합니다. 판단 기준은 (1) 사람별 우선순위, (2) 호출어로 직접 불렀는지, (3) LLM이 판단한 중요도 순입니다. 후보가 2개 이상일 때만 LLM 판단을 하므로 평소에는 지연이 늘지 않습니다. | 새 요청의 점수가 더 높으면 멈추고 새 요청에 답합니다. 낮거나 같으면 끝까지 재생하고, 그 뒤 남은 후보 중 가장 높은 것에 답합니다. |
LLM이 동시에 처리하는 요청 수는 이 정책과 상관없이 채팅 채널 설정을 따릅니다([LLM 동시 요청 수](#llm-동시-요청-수)). 음성 답은 어느 정책이든 한 번에 하나씩 준비하고, 한 답의 글은 말하기보다 훨씬 빨리 끝나므로 다음 답 준비가 밀리지 않습니다.
LLM은 이 정책과 상관없이 요청을 하나씩 처리합니다([LLM 동시 요청 수](#llm-동시-요청-수)). 음성 답은 어느 정책이든 한 번에 하나씩 준비하고, 한 답의 글은 말하기보다 훨씬 빨리 끝나므로 다음 답 준비가 밀리지 않습니다.
버려진 요청과 중간에 멈춘 답은 대시보드의 "놓친 요청" 목록에 남아서, 무엇이 왜 빠졌는지 볼 수 있습니다.
@@ -62,7 +62,7 @@ LLM이 동시에 처리하는 요청 수는 이 정책과 상관없이 채팅
### LLM 동시 요청 수
채팅 채널 중 하나라도 "모든 메시지에 대답"이면 LLM이 요청을 2개 이상(그래픽카드에 맞는 수, 최소 2개) 동시에 처리합니다. 그 채널에서는 사람마다 메시지가 따로 들어오고 각자 글로 답을 받으므로, 한 답이 끝나기를 기다리지 않고 함께 준비해야 하고 음성 답도 채팅 답 뒤로 밀리지 않습니다. 그런 채널이 없으면 음성 응답 정책과 상관없이 1개입니다. 16GB 카드에서는 1개일 때 답이 더 빠릅니다(출력 97 대 70~75 tok/s). 이 수는 LLM을 다시 시작해야 바뀌므로, 첫 "모든 메시지에 대답" 채널을 켜거나 마지막 하나를 끄면 지금 답이 끝난 뒤 다시 시작합니다(약 30초). 게임 중에는 1개입니다([pc-features.md](pc-features.md#자동-성능-조절)).
LLM은 요청을 항상 하나씩 처리합니다. 채팅 채널이 "모든 메시지에 대답"이어도 같아서, 채팅 답과 음성 답은 LLM에서 차례를 기다립니다. 16GB 카드에서는 1개일 때 답이 더 빠르고(출력 97 대 70~75 tok/s), 2개로 둔 날(2026-11-16)에는 프롬프트 읽기가 약 3배 느려져 답이 10~20초 걸렸습니다(사용자 결정, 2026-11-17). 게임 중에도 1개입니다([pc-features.md](pc-features.md#자동-성능-조절)).
- **음성과 연결**: 음성으로 "자비스, 채팅 채널에 오늘 할 일 올려 줘", "채팅에 방금 뭐 올라왔어?"처럼 시킬 수 있습니다.
- **디스코드 제약**: 봇은 자기가 쓴 메시지만 수정할 수 있습니다. 다른 사람 메시지는 삭제·고정만 되고, 서버에서 메시지 관리 권한을 줘야 합니다.

View File

@@ -12,7 +12,7 @@
| 관리 앱 언어 | Rust. 화면(M5)은 Tauri 2로 Windows 기본 WebView2에 그림(2026-11-11, 사용자가 맡긴 판단). 같은 M2 음성 코어를 Electron과 Rust로 둘 다 만들어 실제 디스코드에서 비교함: 받아 적기·말하기·정지 결과는 같았고, Rust는 실행 파일 하나에 메모리 약 16MB(Electron은 프로세스 5개, 크로미움 GPU 프로세스 포함)라 게임과 함께 돌리는 이 앱에 맞음. 게임 감지, 화면 캡처, 입력, 프로세스 우선순위(M7)도 Windows API를 바로 부를 수 있음. 약점은 Rust 디스코드 음성 라이브러리(songbird)가 덜 검증된 점: DAVE 수신 해독 버그는 앱이 직접 해독해 피했고, 그룹이 바뀐 뒤 받기가 멈춘다는 보고(#310)는 들어왔다 나가기 3번 시험에서 재현되지 않음. 몇 시간짜리 시험에서 멈추면 그 부분만 discord.js 보조 프로세스로 돌리는 것을 예비안으로 둠 ([architecture.md](architecture.md#기술-선택과-이유)) |
| 호출 방식 기본값 | 호출어로 부를 때만 응답 |
| 지정한 사람 설정 | "지정한 사람만 응답"과 "지정한 사람은 호출어 없이 대답"을 각각 켜기/끄기. 기본은 둘 다 꺼짐 ([conversation.md](conversation.md#호출어와-호출-방식)) |
| 동시 대화 응답 정책 기본값 | 처음 대화만 |
| 동시 대화 응답 정책 기본값 | 마지막 대화만(사용자 결정, 2026-11-17. 처음에는 처음 대화만) |
| "제일 중요한 대화" 판단 기준 | 사람별 우선순위 → 호출어로 직접 불렀는지 → LLM이 판단한 중요도 |
| 대화 기억 범위 기본값 | 본인 기록 + 등록한 채팅 채널 + 같은 음성 채널에 함께 있던 다른 사람 말. 채팅 채널과 다른 사람 말은 각각 켜기/끄기, 기본은 둘 다 켜짐 ([memory.md](memory.md)) |
| 가상 디스플레이 드라이버 | 필요해지면 설치. 설치 때 관리자 권한 1회. 우리 설치 파일 하나로 어느 PC에서나 돼야 하므로 Parsec 드라이버는 제품에 쓰지 않음(M0 개념 확인용으로만 사용). 대상은 x64 Windows 10/11(S 모드 제외). ARM64는 WHQL 서명이 필요해 범위 밖. 배포할 드라이버는 오픈소스 Virtual Display Driver(MIT, SignPath 재단 서명, IddCx)를 설치 파일에 함께 넣음(사용자 선택). 자체 드라이버는 만들지 않음 |
@@ -35,7 +35,7 @@
|---|---|
| TTS | MeloTTS만 기본 제공. CosyVoice2/3는 첫 소리까지 중앙값 3.4~3.6초라 뺌. 앱 시작 때 한 번 미리 합성(첫 합성 30초) |
| STT | large-v3-turbo, `int8_float16`(모델 1.3GB, 10명 동시 전사 때 1.44GB 더 늘어 최대 약 2.7GB). `float16`과 정확도가 같았음 |
| LLM | Qwen3.6-35B-A3B 하나(위 표). 동시 요청은 채팅 채널 설정을 따름: "모든 메시지에 대답" 채널이 하나라도 있으면 2개 이상(카드에 맞는 수, 최소 2개), 없으면 음성 응답 정책과 상관없이 1개(사용자 결정, 2026-11-10. 16GB 카드에서 1개가 출력 97 tok/s로 2개의 70~75 tok/s보다 빠름). 게임 중에는 채널 설정과 상관없이 1개. 이미지는 GPU 처리, 화질 1280x720, 음성 답변은 생각 모드 끔. 함께 잰 Qwen3.8-Flash-Next(Q2_0, MTP 보조 모델)는 출력 131 tok/s로 82 tok/s보다 빨랐지만 음성 답변 첫 문장(0.58초 대 0.36초), 맥락 8천 토큰 첫 토큰(2.1초 대 1.6초), 이미지 1장 답변 시작(1.3초 대 0.6초)이 모두 느리고, 한국어 답에 한자(礼貌)가 한 번 섞였고, 아래 도구 호출 문제가 있어 뺌 |
| LLM | Qwen3.6-35B-A3B 하나(위 표). 동시 요청은 항상 1개(사용자 결정, 2026-11-17. 16GB 카드에서 1개가 출력 97 tok/s로 2개의 70~75 tok/s보다 빠르고, 2개로 쓴 날 프롬프트 읽기가 약 3배 느려짐. 2026-11-10에는 "모든 메시지에 대답" 채팅 채널이 있으면 2개로 정했었음). 이미지는 GPU 처리, 화질 1280x720, 음성 답변은 생각 모드 끔. 함께 잰 Qwen3.8-Flash-Next(Q2_0, MTP 보조 모델)는 출력 131 tok/s로 82 tok/s보다 빨랐지만 음성 답변 첫 문장(0.58초 대 0.36초), 맥락 8천 토큰 첫 토큰(2.1초 대 1.6초), 이미지 1장 답변 시작(1.3초 대 0.6초)이 모두 느리고, 한국어 답에 한자(礼貌)가 한 번 섞였고, 아래 도구 호출 문제가 있어 뺌 |
| 지난 대화 맥락 | 약 8천 토큰에서 시작(텍스트 첫 문장 1.8초). 상한은 M4에서 왕복 지연을 실측해 정함 |
| 도구 호출 | Strata에서 동작(빠름 12번 중 9번, 권장 12번 중 8번). 둘 다 도구를 부르지 않고 한 척하는 경우가 있음: 빠름 모델은 "5분 타이머 맞춰 줘"에서 "설정했습니다"라고만 답하고(3번 모두), 권장 모델은 "서울 날씨 어때?"에서 날씨를 지어내 답함(3번 모두). 두뇌 코드에서 고침(2026-11-10): (1) 지시문에 "바뀌는 정보(날씨·뉴스·가격·화면)는 이번 턴 도구 결과로만 말하고, 동작은 도구가 성공했을 때만 했다고 말한다"는 규칙을 넣고, 지시문 끝에 이번 턴에 쓸 수 있는 도구 이름을 다시 적음. (2) 모델이 계획 단계가 고른 도구를 건너뛰고 답하면 그 답을 버리고 계획에 적힌 호출을 직접 실행한 뒤 결과로 다시 답하게 함(인자가 도구 형식에 맞을 때만, 아니면 원래 답 유지). (3) 계획 단계에 도구 인자 이름을 보여 줌. 측정 키트 `tools_fast`/`tools_recommended` 단계로 두 모델에서 다시 확인. 수정 후에도 틀리면 모델을 바꿈(사용자 결정): 키트가 같은 계열의 덜 압축된 모델(빠름: UD-Q6_K_XL, Q8_0 / 권장: IQ3_XXS, IQ3_S)을 차례로 재고, 처음 통과한 모델을 기본값으로 함. 재측정 결과(2026-11-10): 빠름 모델 UD-Q4_K_XL이 27번 모두 통과해 기본값 그대로 둠. 도구를 모두 모델이 직접 불렀고 "건너뛴 도구 직접 실행" 장치는 한 번도 돌지 않음. (처음 채점에서는 "할 수 있는 도구는 없습니다/준비되어 있지 않습니다"를 거절로 읽지 못해 6번을 실패로 잘못 매겼고, 그래서 Q6_K_XL과 Q8_0까지 받아 잼. 채점을 고친 뒤 세 모델 모두 27번 통과.) 권장 모델은 Q2_0, IQ3_XXS, IQ3_S 모두 27번 중 24번: "거실 불 꺼 줘"에서 계획 단계는 도구가 필요 없다고 했는데 상관없는 도구(화면 보기, 날씨, 한 번은 0분 타이머)를 부름. 압축 정도와 상관없는 모델 성질이라 권장 모델은 뺌 |
| 분리된 화면공유 창 | 분리 창은 하나뿐이고 그 안에서 볼 화면공유를 고름. 창 제목에는 음성 채널 이름만 있어 누구 화면인지 모름. 누가 공유 중인지는 음성 채널 상태로 확인. 물어본 사람이 공유 중이 아니면 그렇다고 답함. 공유 중이면 공유자가 여러 명이어도 따로 묻지 않고 분리 창을 바로 캡처(사용자가 이미 그 사람 화면을 보고 있음). 가려져도 영상은 계속 그려짐 |

View File

@@ -25,8 +25,8 @@
| 단계 | 언제 | LLM 배치 (다시 시작 필요) | 다시 시작 없이 바꾸는 것 |
|---|---|---|---|
| 평소 | 게임이 없고 VRAM이 넉넉함 | 전문가 층의 VRAM 몫(`--expert-vram`)을 STT 몫과 다른 프로그램용 여유(기본 1GB)를 남기고 계산해서 정함(아래 식, 게임용으로 비울 VRAM은 0). Strata 자동 배치는 STT 자리를 남기지 않아 쓰지 않음(아래 측정 결과). 이미지는 GPU로 읽기. 동시 요청 수는 [채팅 채널 설정](conversation.md#llm-동시-요청-수)을 따름: "모든 메시지에 대답" 채널이 하나라도 있으면 2개 이상(이 카드에 맞는 수, 최소 2개), 없으면 1개(하나씩 보면 더 빠름) | 화면 캡처 미리 준비(말하는 중), 화질 보통, 지난 대화 약 8천 토큰 |
| 게임 중 | 게임을 감지했거나, 다른 프로그램이 VRAM을 써서 남은 VRAM이 "게임용으로 비울 VRAM"보다 적음 | 전문가 층의 VRAM 몫(`--expert-vram`)을 줄여 "게임용으로 비울 VRAM"(기본 8GB)을 비움. 나머지 전문가 층은 RAM에서 CPU로 계산. 동시 요청 1개(채팅 채널 설정과 상관없이), 이미지는 CPU로 읽기, 엔진 CPU 작업자는 절반(16코어 PC에서 8개), 맥락 16K. 대화 메모리 형식(`--kv`)과 MTP 끄기는 이 엔진에서 VRAM을 줄이지 않아 쓰지 않음(아래 측정 결과) | 캡처 미리 준비 끔, 화질 빠름(긴 변 768px), 지난 대화 약 4천 토큰 |
| 평소 | 게임이 없고 VRAM이 넉넉함 | 전문가 층의 VRAM 몫(`--expert-vram`)을 STT 몫과 다른 프로그램용 여유(기본 1GB)를 남기고 계산해서 정함(아래 식, 게임용으로 비울 VRAM은 0). Strata 자동 배치는 STT 자리를 남기지 않아 쓰지 않음(아래 측정 결과). 이미지는 GPU로 읽기. 동시 요청 1개([LLM 동시 요청 수](conversation.md#llm-동시-요청-수), 하나씩 보면 더 빠름) | 화면 캡처 미리 준비(말하는 중), 화질 보통, 지난 대화 약 8천 토큰 |
| 게임 중 | 게임을 감지했거나, 다른 프로그램이 VRAM을 써서 남은 VRAM이 "게임용으로 비울 VRAM"보다 적음 | 전문가 층의 VRAM 몫(`--expert-vram`)을 줄여 "게임용으로 비울 VRAM"(기본 8GB)을 비움. 나머지 전문가 층은 RAM에서 CPU로 계산. 동시 요청 1개, 이미지는 CPU로 읽기, 엔진 CPU 작업자는 절반(16코어 PC에서 8개), 맥락 16K. 대화 메모리 형식(`--kv`)과 MTP 끄기는 이 엔진에서 VRAM을 줄이지 않아 쓰지 않음(아래 측정 결과) | 캡처 미리 준비 끔, 화질 빠름(긴 변 768px), 지난 대화 약 4천 토큰 |
| 일시정지 | 사용자가 켠 경우(기본 꺼짐), 또는 이 PC에서 최소 배치로도 게임 몫을 비울 수 없을 때 | LLM을 멈춤 | 게임 중에는 대답하지 않음 |
### PC마다 값을 계산
@@ -34,7 +34,7 @@
- **배치는 계산해서 정함**: 전문가 층 VRAM 몫 = 그래픽카드 VRAM − 게임용으로 비울 VRAM − 다른 프로그램이 이미 쓰는 VRAM − STT 몫(약 2.7GB: 모델 1.3GB와 10명 동시 전사 때 늘어나는 1.44GB, M0 `stt` 실측) − 엔진 기본 버퍼 − 여유분(768MiB, 엔진이 스스로 남기는 양과 같음). 평소 배치도 같은 식이고, 게임용으로 비울 VRAM 대신 다른 프로그램용 여유(기본 1GB)를 뺍니다(사용자 결정, 2026-11-10). 이 여유는 나중에 켜는 브라우저 영상 같은 프로그램 몫이고, 이미 켜져 있는 프로그램은 "다른 프로그램이 이미 쓰는 VRAM"으로 따로 빠집니다. 게임 중에는 게임용으로 비울 VRAM이 이 역할을 하므로 다른 프로그램용 여유를 따로 빼지 않습니다. 엔진 기본 버퍼는 첫 실행 자가 점검에서 전문가 층 없이 한 번 켜서 잽니다. 정해 둔 숫자가 아니라 이 식으로 구하므로 VRAM이 작은 PC에도 같은 규칙이 맞습니다.
- **게임 중 최소 배치는 전문가 2GB입니다**(사용자 결정, 2026-11-10). 전문가 몫에는 긴 프롬프트용 버퍼 1.1GiB가 먼저 들어가므로 2GB는 전문가를 실제로 올리는 가장 작은 값이고, 실제 게임 중에는 0GB보다 출력이 1.5~2.1배 빨랐습니다(아래 측정 결과). 0GB는 2GB로는 게임 몫이 0 이하가 되는 PC에서만 일시정지 전 마지막 단계로 씁니다.
- **몫이 최소 배치보다 작을 때**: 계산한 몫이 최소 배치(전문가 2GB)보다 작으면, 몫을 늘려 게임 몫을 침범하지 않습니다. 대신 게임용으로 비울 VRAM을 "그래픽카드 VRAM − 다른 프로그램 − STT 몫 − 엔진 기본 버퍼 − 최소 배치"까지 줄이고, 대시보드와 게임 모드 설정 화면에 "이 PC에서는 게임에 X GB까지만 비울 수 있음"을 표시합니다. 이렇게 줄인 값이 0 이하이면(예: 8GB 그래픽카드에서 다른 프로그램이 이미 많이 씀) 게임 중에는 LLM을 멈추고 일시정지 단계로 갑니다. 게임 중 일시정지를 꺼 두었어도 마찬가지이고, 그 이유를 대시보드에 표시합니다. 게임이 끝나면 되돌리기 규칙대로 돌아옵니다.
- **첫 실행 자가 점검**: 설치가 끝나면 평소·게임 중 배치로 각각 30초씩 출력 속도와 음성 첫 문장 시간을 잽니다. 평소 배치가 기준(출력 30 tok/s, 음성 첫 문장 0.7초)에 못 미치는 PC는 평소에도 동시 요청 1개, 이미지 CPU 읽기, 지난 대화 약 4천 토큰으로 시작합니다. 게임 중 배치가 기준에 못 미치면 대시보드와 설정 화면에 "게임 중에는 답이 느릴 수 있음"을 표시합니다.
- **첫 실행 자가 점검**: 설치가 끝나면 평소·게임 중 배치로 각각 30초씩 출력 속도와 음성 첫 문장 시간을 잽니다. 평소 배치가 기준(출력 30 tok/s, 음성 첫 문장 0.7초)에 못 미치는 PC는 평소에도 이미지 CPU 읽기, 지난 대화 약 4천 토큰으로 시작합니다. 게임 중 배치가 기준에 못 미치면 대시보드와 설정 화면에 "게임 중에는 답이 느릴 수 있음"을 표시합니다.
- **GPU 사용률 제한은 두지 않습니다**: Jarvis 옆에서 보는 영상이 끊기지 않도록 LLM 엔진과 음성·두뇌 워커를 200ms마다 함께 10ms씩 멈추는 제한(Windows 프로세스 일시정지)을 검토했지만 뺐습니다(2026-11-11). M0 `gamemode_tune`에서는 멈춘 시간이 정한 대로 5.2%였는데도 nvidia-smi 사용률이 내려가지 않았고(STT를 쉬지 않고 돌릴 때 중앙값 96→98%, LLM만 40→39%) 출력만 6~10% 느려졌습니다([측정 결과](#자동-조절-측정-m0-gamemode_tune-2026-11-11)). M0 `video_drops`에서는 제한이 없어도 Jarvis가 GPU를 98%까지 쓰는 동안 유튜브 1080p 60fps 영상이 끊기지 않았고, 95% 제한은 끊긴 프레임을 줄이지 못했습니다([측정 결과](#영상-끊김-측정-m0-video_drops-2026-11-11)). 미리 정한 기준(제한이 끊김을 줄이지 못하면 설정을 뺌)대로 설정에서 뺐습니다. 게임 화면 프레임에 주는 영향은 아래 `game_frames`로 따로 잽니다.
- **대신 게임 화면의 프레임을 직접 봤고, 양보는 넣지 않습니다(M0 `game_frames`, 2026-11-11)**: 사용률 제한은 GPU 전체 사용률을 보고 Jarvis를 일정 시간씩 멈추는 방식이라, 게임이 GPU를 원하지 않을 때도 Jarvis를 느리게 했고 정작 막으려던 것(화면이 끊기는 것)은 보지 않았습니다. 그래서 게임 화면의 프레임을 기준으로 삼고, 멈추는 대신 우선순위로 양보하는 방법을 후보로 함께 쟀습니다. 실제 게임 중 Jarvis가 바쁠 때 평균 fps는 0.2%, 1% low는 3.3% 떨어졌고 50ms 넘는 프레임은 늘지 않아서(0개), 아래 미리 정한 기준대로 양보는 넣지 않습니다([측정 결과](#게임-프레임-측정-m0-game_frames-2026-11-11)).
- **기준은 게임 화면의 프레임 시간입니다**: 평균 fps, 1% low(가장 느린 1% 프레임), 50ms·100ms를 넘는 프레임 수를 봅니다. 게임이 프레임을 만든 간격이 아니라 화면이 실제로 바뀐 간격으로 셉니다. 게임은 60fps로 프레임을 내도 화면에는 일부만 나갈 수 있기 때문입니다. 화면에 나가지 못한 프레임 수도 따로 셉니다. Windows가 기록하는 프레임 시각만 읽고 게임 화면은 보지 않습니다.
@@ -45,7 +45,6 @@
### 전환 규칙
- **게임 시작**: 지금 하던 답이 끝나면 바로 게임 중 배치로 다시 시작합니다. 다시 시작하는 동안(게임PC 측정 31~36초, 컴퓨터를 켠 뒤 첫 시작은 47초) 들어온 요청은 대기열에 두고, 한 번만 "잠깐만요, 게임 모드로 바꾸는 중이에요"라고 말한 뒤 준비되면 답합니다.
- **채팅 채널 설정을 바꿀 때**: 동시 요청 수가 바뀌는 변경(첫 "모든 메시지에 대답" 채널을 켜거나 마지막 하나를 끔)은 LLM을 다시 시작해야 합니다. 게임 시작과 같이 지금 하던 답이 끝난 뒤 다시 시작하고, 그동안 들어온 요청은 대기열에 둡니다. 게임 중에는 다시 시작하지 않고 게임이 끝나 평소 배치로 돌아갈 때 함께 반영합니다.
- **되돌리기**: 게임이 끝나고 30초 뒤(설정에서 조절), VRAM 부족은 해소된 상태가 60초 이어진 뒤 평소 배치로 돌아갑니다. 게임을 껐다 바로 다시 켤 때 모델을 반복해서 옮기지 않기 위해서입니다.
- **표시**: 대시보드에 "성능 단계: 게임 중 (발로란트 감지)", 전환 중에는 "모델 배치 바꾸는 중"을 표시합니다.
- **설정**: GUI에서 바꾸는 것은 게임 감지 켜기/끄기, 게임용으로 비울 VRAM, 다른 프로그램용 여유 VRAM(평소, 기본 1GB), 되돌리기 대기 시간, 게임 중 일시정지 켜기/끄기뿐입니다.
@@ -83,7 +82,7 @@
- **이 PC의 게임 중 배치**: 기본값 8GB는 비울 수 없으므로 [몫이 최소 배치보다 작을 때](#pc마다-값을-계산) 규칙대로 최소 배치(전문가 2GB, 맥락 16K)로 갑니다. LLM과 STT(GPU, 10명 동시 전사)를 함께 켜고 잰 게임 몫은 2.6GB여서(아래 LLM과 STT를 함께 켠 측정) "이 PC에서는 게임에 약 2.6GB까지만 비울 수 있음"을 표시합니다. 0GB 배치(게임 몫 3.6GB)는 실제 게임에서 출력이 2GB의 절반 정도라 쓰지 않습니다(사용자 결정, 2026-11-10).
- **CPU 작업자**를 16개에서 8개로 줄여도 출력 속도가 줄지 않았습니다(73.8 대 75.8 tok/s). 게임 중에는 8개로 둡니다.
- **STT는 게임 중에도 GPU에 둡니다.** CPU로 옮기면 게임 몫이 약 2.7GB 늘고 LLM도 빨라지지만, 4~5초짜리 말 한 마디를 받아 적는 데 3.0~3.8초(GPU 0.11~0.19초), 10명 동시 전사 한 번에 32~37초(GPU 1.1~3.0초)가 걸려 0.3~0.5초마다 하는 부분 전사를 따라가지 못합니다. 실제 게임 중 전문가 2GB 배치에서 한 사람 전사와 짧은 프롬프트의 음성 첫 문장 중앙값(서로 다른 측정)을 단순히 더하면 CPU 약 4.3초, GPU 약 1.2초입니다. TTS와 전사 대기는 빠진 추정이라 실제 음성 시작 시간은 이보다 깁니다.
- **평소 동시 요청 수**: 동시 2개는 Strata가 "16GB 카드에는 1개가 맞음"이라고 경고하고, 전문가 층을 덜 올려서 동시 1개 배치(전문가 6GB)보다 출력이 느렸습니다(69.6~74.7 대 97.1 tok/s). 그래서 평소에는 1개로 두고, 채팅 채널 중 하나라도 "모든 메시지에 대답"일 때만 2개 이상으로 합니다(사용자 결정, 2026-11-10).
- **평소 동시 요청 수**: 동시 2개는 Strata가 "16GB 카드에는 1개가 맞음"이라고 경고하고, 전문가 층을 덜 올려서 동시 1개 배치(전문가 6GB)보다 출력이 느렸습니다(69.6~74.7 대 97.1 tok/s). 그래서 평소에도 1개로 둡니다. 처음에는 채팅 채널 중 하나라도 "모든 메시지에 대답"이면 2개로 했지만(2026-11-10), 그렇게 쓴 날 프롬프트 읽기가 약 3배 느려져 항상 1개로 바꿨습니다(사용자 결정, 2026-11-17).
세 번째 실행(LLM만, 게임 중 후보는 모두 동시 1개, 이미지 CPU, CPU 작업자 8개):

View File

@@ -36,7 +36,7 @@ def warm_up_ollama_model(base_url: str, model: str, timeout: float) -> bool:
if is_openai_compatible(base_url):
# The server loads its model at launch; one token checks it answers.
body = openai_chat_body(model, [{"role": "user", "content": "hi"}], stream=False,
thinking=False, max_tokens=1)
thinking=False, max_tokens=1, one_shot=True)
try:
response = post_chat_completion(base_url, body, timeout)
ok = response.status_code == 200
@@ -438,6 +438,7 @@ Examples:
self.config.model,
[{"role": "system", "content": system_prompt}, {"role": "user", "content": user_prompt}],
stream=False, thinking=self.config.thinking, max_tokens=200, temperature=0.0,
one_shot=True,
),
self.config.timeout_sec,
)

View File

@@ -110,7 +110,11 @@ def openai_chat_body(
max_tokens: Optional[int] = None,
temperature: Optional[float] = None,
tools: Optional[List[Dict[str, Any]]] = None,
one_shot: bool = False,
) -> Dict[str, Any]:
"""``one_shot``: no later request continues this one (a classification, an extraction, a
warm-up). Strata then gives its session away first, so a reply's tool turn keeps its own
session through the short calls made between its steps; other servers ignore the field."""
body: Dict[str, Any] = {
"model": chat_model,
"messages": _openai_messages(messages),
@@ -124,6 +128,8 @@ def openai_chat_body(
body["temperature"] = temperature
if tools:
body["tools"] = tools
if one_shot:
body["strata_checkpoint"] = False
return body
@@ -246,7 +252,8 @@ def call_llm_direct(base_url: str, chat_model: str, system_prompt: str, user_con
if is_openai_compatible(base_url):
caller = _caller_name()
body = openai_chat_body(chat_model, messages, stream=False, thinking=thinking, temperature=temperature)
body = openai_chat_body(chat_model, messages, stream=False, thinking=thinking, temperature=temperature,
one_shot=True)
try:
content = _openai_chat(base_url, body, timeout_sec, caller)["message"]["content"]
if content.strip():
@@ -386,7 +393,7 @@ def _openai_stream(
thinking: bool,
) -> Optional[str]:
"""Server-sent-events stream from ``/chat/completions``; answer text only (no reasoning)."""
body = openai_chat_body(chat_model, messages, stream=True, thinking=thinking)
body = openai_chat_body(chat_model, messages, stream=True, thinking=thinking, one_shot=True)
try:
with post_chat_completion(base_url, body, timeout_sec, stream=True) as resp:
resp.raise_for_status()

View File

@@ -13,6 +13,7 @@ from ..system_prompt import (
build_system_prompt,
build_voice_persona,
COMPACT_ACTION_RULE,
COMPACT_TOOL_GUIDANCE,
load_agent_instructions,
reply_language_directive,
tool_honesty_reminder,
@@ -1095,6 +1096,23 @@ def _attach_images(message: dict, result) -> None:
message["images"] = list(images)
def _starts_with_text_tools(cfg) -> bool:
"""Whether the reply starts in text-tool mode: a small model outside the native tool-call
families (see the loop's comment where it is used)."""
model_l = (cfg.ollama_chat_model or "").lower()
native_capable = any(k in model_l for k in (
"qwen2.5", "qwen2", "qwen3", "llama3.1", "llama3.2", "llama3.3",
"mistral", "hermes", "command-r", "firefunction",
))
return detect_model_size(cfg.ollama_chat_model) == ModelSize.SMALL and not native_capable
# Tools that do their job in one call (planner.spec.md, "Intent judge (v2)"): a v2 turn whose only
# router pick is one of them needs no plan. Tools a request may use several times or chain (search,
# browsers, schedule and notes lists before a change, chat channels) are not here.
_ONE_STEP_TOOLS = frozenset({"getWeather", "logMeal", "fetchMeals", "screenView"})
def _offered_builtins(cfg, chat_available: bool, schedule_available: bool = False,
notes_available: bool = False, screen_available: bool = False) -> dict:
"""The built-in tools this turn can offer: the enabled ones (``enabled_builtin_tools``),
@@ -1400,6 +1418,14 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# aside; the chat tool counts here, its every-turn offer is added below); a turn without one
# goes straight to its answer.
_turn_needs_tools = any(t not in ("stop", "toolSearchTool") for t in routed_tools)
# Intent judge (v2), native tool calls: a lone one-step tool goes to the chat model without a
# plan; the nudge in the loop below stands in for the plan-based skipped-tool guard.
_picked_tools = [t for t in routed_tools if t not in ("stop", "toolSearchTool")]
_one_step_tool = (
_picked_tools[0]
if intent_judge and len(_picked_tools) == 1 and _picked_tools[0] in _ONE_STEP_TOOLS
and not _starts_with_text_tools(cfg) else None
)
# In screen-share mode, always offer setBroadcast. "Turn the broadcast
# on/off" is language-agnostic intent the embedding/keyword router won't
@@ -1447,7 +1473,9 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
try:
# Intent judge (v2): the router decides whether the turn needs tools; only then does the
# planner order the steps (planner.spec.md, "Intent judge (v2)").
if not intent_judge or _turn_needs_tools:
if _one_step_tool:
debug_log(f"intent judge: one-step tool {_one_step_tool}, no plan", "planning")
elif not intent_judge or _turn_needs_tools:
action_plan = plan_query(
cfg=cfg,
query=redacted,
@@ -1866,12 +1894,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# call, so getWeather/webSearch/controlBrowser never run. Use native for the
# tool-capable families (native still auto-falls-back to text on HTTP 400);
# only genuinely non-tool small models (e.g. gemma) default to text.
_model_l = (cfg.ollama_chat_model or "").lower()
_native_capable = any(k in _model_l for k in (
"qwen2.5", "qwen2", "qwen3", "llama3.1", "llama3.2", "llama3.3",
"mistral", "hermes", "command-r", "firefunction",
))
use_text_tools = (model_size == ModelSize.SMALL) and not _native_capable
use_text_tools = _starts_with_text_tools(cfg)
prompts = get_system_prompts(model_size)
debug_log(f"Model size detected: {model_size.value} for {cfg.ollama_chat_model} (use_text_tools={use_text_tools})", "planning")
@@ -1919,9 +1942,10 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# The persona already covers voice style, inference and tool use; the tool guidance
# (named entities, auto-derived and self-contained arguments) matters only when a
# data tool is on offer. The sentinels and the always-offered chat tool are not.
# It is v1's guidance in short (COMPACT_TOOL_GUIDANCE): every token is read each reply.
guidance.append(prompts.asr_note)
if any(t not in ("stop", "toolSearchTool", "chatChannels") for t in allowed_tools):
guidance.extend(c for c in (prompts.tool_guidance, prompts.tool_constraints) if c)
guidance.append(COMPACT_TOOL_GUIDANCE)
else:
guidance.append(COMPACT_ACTION_RULE)
else:
@@ -2281,6 +2305,9 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# synthesise from the real result. Fires at most once per reply; a
# non-empty list means it fired. See planner.spec.md.
_forced_plan_steps: list[str] = []
# One-step turns have no plan: an answer given without their tool is dropped and the model is
# asked once to call it (planner.spec.md, "Intent judge (v2)").
_one_step_nudged = False
while turn < max_turns:
turn += 1
@@ -3002,6 +3029,36 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
candidate_reply = content
malformed_fallback = False
_one_step_skipped = bool(
_one_step_tool and _one_step_tool in allowed_tools and not use_text_tools
and not _forced_plan_steps and not malformed_fallback
and sum(1 for m in messages if m.get("tool_name")) == _plan_steps_baseline
)
if _one_step_skipped and not _one_step_nudged:
messages.pop()
messages.append({"role": "user", "content": (
f"[Before answering, call {_one_step_tool}: this request needs its result. "
"If a detail it needs is missing, ask the user for it instead.]")})
_one_step_nudged = True
debug_log(f"one-step turn: answered without {_one_step_tool}; asking for the call", "planning")
continue
if _one_step_skipped and not action_plan:
# Asked once and still no tool: this turn takes the planned path after all, so the
# skipped-tool guard below runs the tool the plan names (or the plan says no tool is
# needed, a question back to the user say, and the answer goes out).
try:
action_plan = plan_query(
cfg=cfg, query=redacted, dialogue_context=_dialogue_ctx,
tools=_planner_tool_catalog, live_context=_live_time_location_string(cfg),
)
except Exception as _plan_exc: # pragma: no cover — defensive
debug_log(f"planner step failed (non-fatal): {_plan_exc}", "planning")
action_plan = []
if action_plan:
_turn_state.trace("plan", steps=list(action_plan))
debug_log(f"one-step turn: still no {_one_step_tool} after the nudge; planned "
f"{len(action_plan)} step(s)", "planning")
_guard_steps = [] if (use_text_tools or _forced_plan_steps or malformed_fallback) else (
named_tool_steps(
action_plan,

View File

@@ -222,6 +222,29 @@ needs the planner to name the channel); its every-turn offer, added after the ro
to its answer with one LLM call before it; a tool turn keeps the planner's ordered steps, which
multi-step asks need ("who directed Possessor, and what else did they make").
**One-step turns.** When the router's only pick (sentinels aside) is a tool that does its job in one
call, `_ONE_STEP_TOOLS` in `engine.py` (`getWeather`, `logMeal`, `fetchMeals`, `screenView`), and the
chat model calls tools natively, the planner does not run: its steps would only repeat the one call
the chat model makes itself. The rule reads the router's pick, never the words, so it is language-
agnostic. Tools a request may use several times or chain stay planned: search and page fetches,
the browsers, `schedule` and `personalNotes` (a list before a change), `chatChannels`. Two or more
picked tools are planned as before. Text-tool mode (small non-native models) always plans, since its
direct execution runs the plan.
Without a plan the skipped-tool guard above has nothing to run, so these turns get a nudge instead:
when the chat model answers in words before any tool ran in this reply, the answer is dropped (no
later turn sees it) and a note asks the model to call that tool first, or to ask the user for a
missing detail. The nudge is sent at most once per reply. When the model answers without the tool
again, that turn falls back to the planned path: the planner runs then (one extra planner call, only
on this failure), and the skipped-tool guard above treats the second answer as on any planned turn.
A plan naming the tool runs it and the made-up answer never goes out; a plan that needs no tool (a
question back to the user, say) lets the answer go out, the planner judging that as before.
Measured on Strata (Qwen3.6, 2026-11-16 16:45-16:58, the engine log): the planner averaged 2.5 s
(9 calls), one after the other before the answer on every tool turn, while the router runs during
speech (`prepare_intent`). A planner call that does not run also leaves the engine's second session
to the chat model, whose prompt read is the largest cost.
Measured on Strata (Qwen3.6, 2026-11-12): the router took 0.42 s and the planner 0.49-0.94 s, one
after the other, before every answer; each call costs at least one full prompt read. Tried first
and dropped: the planner alone over the whole catalogue (one call) planned "reply with the

View File

@@ -106,7 +106,7 @@ Design principles enforced by the engine:
- When detected, the engine falls back to the standard "I had trouble understanding that request" error reply (model-size-aware). The malformed content is never shown to the user.
Skipped-tool guard (native tool calling):
- When a native-tool model answers in prose before any tool has run in this reply, and the plan names a callable tool as a concrete call, the engine discards that answer and direct-executes the plan's tool steps, then the model answers from the results. Only literal, schema-valid steps run (no LLM repair). When the planned call cannot run, the held-back answer is still discarded: the planned call is recorded as a failed tool result and the model answers from that (an honest "could not check or do it"). See `planner.spec.md` ("Skipped-tool guard").
- When a native-tool model answers in prose before any tool has run in this reply, and the plan names a callable tool as a concrete call, the engine discards that answer and direct-executes the plan's tool steps, then the model answers from the results. Only literal, schema-valid steps run (no LLM repair). When the planned call cannot run, the held-back answer is still discarded: the planned call is recorded as a failed tool result and the model answers from that (an honest "could not check or do it"). See `planner.spec.md` ("Skipped-tool guard"). v2 one-step turns (no plan) get one nudge to call the tool instead, and when the model skips it again the turn is planned and guarded as above (`planner.spec.md`, "Intent judge (v2)").
Task-list planner (all model sizes, strongest impact on small models):
- The planner runs at the **front** of the reply flow (see step 3 above), not after tool selection. By the time the agentic loop starts, the plan already exists, the memory block has either been run or skipped based on the plan's `searchMemory` directive, and the tool allow-list has been derived from the tool names the plan referenced. See `planner.spec.md` for the prompt contract and fail-open semantics.

View File

@@ -209,6 +209,27 @@ COMPACT_ACTION_RULE: str = (
)
# Compact prompts, turns with a data tool: v1's tool guidance and constraints (about 900 Qwen3.6
# tokens) in a quarter of the words. Strata reads prompts in 2,048-token steps of about 0.4 s each,
# so these tokens decide whether a tool turn with recent conversation stays inside one step fewer.
# The same rules as v1's (acting instead of refusing, facts only from the result, looking up named
# things, auto-derived and self-contained arguments), without English examples.
COMPACT_TOOL_GUIDANCE: str = (
"Using tools: when a tool here covers the request, call it straight away, without announcing it "
"or asking permission; when none fits an action or a live fact, call toolSearchTool with a short "
"description before saying you cannot. Answer from tool results, never about them: every "
"specific fact in the reply (names, dates, numbers, places, cast, plot) must come from the tool "
"result or the user's own words, not from your prior knowledge; if the result lacks it, say so. "
"When a search result has a 'Content from top result' section, give its facts, not the links. "
"A specific named thing (a film, song, game, product, person, place or event) you cannot state "
"verified facts about is looked up before you answer. When a tool fills an argument itself (for "
"example the current location), call it with what you have instead of asking; a value the user "
"already gave in the conversation is passed. Free-text arguments such as search queries must "
"stand on their own, because the tool does not see earlier turns: resolve pronouns and "
"references into a short keyword phrase."
)
def tool_honesty_reminder(tool_names) -> str:
"""End-of-prompt restatement of the tool honesty rules for this turn.
@@ -307,14 +328,12 @@ def default_language_directive(language: Optional[str]) -> Optional[str]:
lang = (language or "").strip()
if not lang:
return None
# Short on purpose: the reply engine puts it at the front and the end of every v2 prompt.
return (
f"REPLY LANGUAGE: write your reply in {lang} by default, whatever language the user's "
f"message or any reference text is in. The one exception overrides this default: when "
f"the user explicitly asks, in any language, for the answer in another language (for "
f"example 'answer in English'), reply in that language, for just this reply or from now "
f"on in this conversation, as they asked. Do not mix in other scripts: no Chinese/Hanja characters, "
f"Japanese kana or Cyrillic letters inside a {lang} reply. (Numerals and unavoidable "
f"proper-noun brand names are fine.)"
f"REPLY LANGUAGE: reply in {lang}, whatever language the message or any reference text is "
f"in, unless the user explicitly asks, in any language, for another language: then use that "
f"one, for this reply or from then on, as they asked. No Chinese/Hanja, Japanese kana or "
f"Cyrillic inside a {lang} reply; numerals and brand names are fine."
)

View File

@@ -267,9 +267,10 @@ which carry Discord's own time.
Two reply options keep the time to the first word short (Strata reads every prompt from the
start, about 0.36 s per 2,048 tokens, and each LLM call before the answer adds its whole time):
the short voice persona (`compact_prompt=True`: the same butler in a tenth of v1's words, without
v1's single-user memory and on-screen browser rules, answers as short as possible, the full tool
guidance only on turns that offer a data tool), and the intent judge (`intent_judge=True`: the tool
router judges whether the turn needs tools and which, and the planner runs only when it does;
v1's single-user memory and on-screen browser rules, answers as short as possible, v1's tool
guidance in short only on turns that offer a data tool), and the intent judge (`intent_judge=True`: the tool
router judges whether the turn needs tools and which, and the planner runs only when it does
and the pick is not a lone one-step tool;
`planner.spec.md`, "Intent judge (v2)"). v1's screen-share browser (`stream_browser`) is off: it
needs v1's Node bot (M6 brings the v2 one), so its tools are not offered.
- Tools: the brain offers and runs no built-in tool in the jarvis settings' `disabled_tools` (the

View File

@@ -2,7 +2,7 @@
Strata reads every prompt from the start, so prompt length is reply latency. Behaviour: the compact
prompt is far shorter than v1's for the same turn, still carries the tool honesty rules and the
reply-language default, and brings the full tool guidance only on turns that offer a data tool.
reply-language default, and brings the (short) tool guidance only on turns that offer a data tool.
"""
from unittest.mock import Mock, patch
@@ -13,7 +13,9 @@ from src.jarvis.memory.conversation import DialogueMemory
from src.jarvis.memory.history import HistoryContext
from src.jarvis.reply.engine import run_reply_engine
from src.jarvis.reply.prompts.model_variants import TOOL_CONSTRAINTS_LARGE
from src.jarvis.system_prompt import COMPACT_ACTION_RULE, TOOL_HONESTY_RULES, default_language_directive
from src.jarvis.system_prompt import (
COMPACT_ACTION_RULE, COMPACT_TOOL_GUIDANCE, TOOL_HONESTY_RULES, default_language_directive,
)
from tests.test_ollama_num_predict import _mock_cfg
@@ -46,14 +48,23 @@ def test_the_compact_prompt_is_far_shorter_and_keeps_the_rules(routed):
@pytest.mark.unit
def test_the_full_tool_guidance_comes_only_with_a_data_tool():
def test_the_tool_guidance_comes_only_with_a_data_tool():
chatty = _system_prompt(True, [])
assert COMPACT_ACTION_RULE in chatty and TOOL_CONSTRAINTS_LARGE not in chatty
assert COMPACT_ACTION_RULE in chatty and COMPACT_TOOL_GUIDANCE not in chatty
weather = _system_prompt(True, ["getWeather"])
assert TOOL_CONSTRAINTS_LARGE in weather and COMPACT_ACTION_RULE not in weather
assert COMPACT_TOOL_GUIDANCE in weather and COMPACT_ACTION_RULE not in weather
def _intent_turn(prepared, routed=("getWeather",)):
@pytest.mark.unit
def test_a_data_tool_turn_gets_the_short_tool_guidance_not_v1s():
# v1 keeps its full guidance; the compact prompt says the same rules in a fraction of it.
assert TOOL_CONSTRAINTS_LARGE in _system_prompt(False, ["getWeather"])
weather = _system_prompt(True, ["getWeather"])
assert TOOL_CONSTRAINTS_LARGE not in weather
assert len(COMPACT_TOOL_GUIDANCE) < len(TOOL_CONSTRAINTS_LARGE) / 2
def _intent_turn(prepared, routed=("getWeather",), first=None):
cfg = _mock_cfg(512)
cfg.stream_browser = False
cfg.tool_selection_strategy = "llm"
@@ -62,7 +73,9 @@ def _intent_turn(prepared, routed=("getWeather",)):
patch("src.jarvis.reply.engine._extra_config", lambda key, default="": default), \
patch("src.jarvis.reply.engine.extract_text_from_response", return_value="네."), \
patch("src.jarvis.reply.engine.chat_with_messages") as mock_chat:
mock_chat.return_value = {"message": {"content": "네."}}
answer = {"message": {"content": "네."}}
mock_chat.side_effect = [first, answer] if first else None
mock_chat.return_value = answer
run_reply_engine(db=Mock(), cfg=cfg, tts=None, text="날씨 어때?", dialogue_memory=DialogueMemory(),
history=HistoryContext(), legacy_memory=False, intent_judge=True,
prepared_tools=prepared)
@@ -74,9 +87,16 @@ def _intent_turn(prepared, routed=("getWeather",)):
def test_the_intent_judge_plans_only_turns_that_need_a_tool():
# No tool needed: one router call, then the answer.
assert _intent_turn(None, routed=())[:2] == (0, 1)
# A tool turn: the planner orders the steps.
planner, router, tools = _intent_turn(None)
assert (planner, router) == (1, 1) and "getWeather" in tools
# A tool turn that may take several steps: the planner orders them.
planner, router, tools = _intent_turn(None, routed=("webSearch",))
assert (planner, router) == (1, 1) and "webSearch" in tools
# A lone one-step tool the model calls needs no plan (tests/test_engine_one_step_turn.py).
weather_call = {"message": {"content": "", "tool_calls": [
{"id": "c1", "type": "function", "function": {"name": "getWeather", "arguments": {}}}]}}
with patch("src.jarvis.reply.engine.run_tool_with_retries",
return_value=Mock(success=True, reply_text="13C", error_message=None, images=None)):
planner, router, tools = _intent_turn(None, first=weather_call)
assert (planner, router) == (0, 1) and "getWeather" in tools
@pytest.mark.unit

View File

@@ -0,0 +1,145 @@
"""One-step tool turns in the v2 brain skip the planner (planner.spec.md, "Intent judge (v2)").
When the router's only pick is a tool that does its job in one call (the weather, a meal log, a
look at the screen), the planner's ordered steps add nothing but a full LLM call before the answer.
The engine then gives the chat model the tool directly. Its skipped-tool guard, which reads the
plan, is replaced for these turns by one nudge: an answer given without the tool is dropped and the
model is asked once to call it.
"""
from __future__ import annotations
from unittest.mock import patch
import pytest
NATIVE_MODEL = "gpt-oss:20b"
FABRICATED = "서울은 지금 24도에 맑습니다."
GROUNDED = "서울은 지금 13도이고 맑습니다."
def _content(text: str) -> dict:
return {"message": {"role": "assistant", "content": text}}
def _native_call(name: str, args: dict) -> dict:
return {"message": {"role": "assistant", "content": "", "tool_calls": [
{"id": "call_1", "type": "function", "function": {"name": name, "arguments": args}}]}}
def _run(mock_config, db, dialogue_memory, *, routed, chat_replies, plan=("Reply to the user.",)):
from jarvis.memory.history import HistoryContext
from jarvis.reply import engine as engine_mod
from jarvis.tools.types import ToolExecutionResult
mock_config.ollama_chat_model = NATIVE_MODEL
mock_config.evaluator_enabled = False
mock_config.stream_browser = False
seen: list[list[dict]] = []
replies = iter(chat_replies)
def fake_chat(*args, **kwargs):
seen.append([dict(m) for m in (kwargs.get("messages") or args[2])])
return next(replies)
invoked: list[str] = []
def fake_tool(db, cfg, tool_name, tool_args, **kwargs):
invoked.append(tool_name)
return ToolExecutionResult(success=True, reply_text="Seoul: 13C, clear sky.", error_message=None)
with patch.object(engine_mod, "run_tool_with_retries", side_effect=fake_tool), \
patch.object(engine_mod, "chat_with_messages", side_effect=fake_chat), \
patch.object(engine_mod, "select_tools", return_value=list(routed)), \
patch.object(engine_mod, "plan_query", return_value=list(plan)) as planner:
reply = engine_mod.run_reply_engine(
db=db, cfg=mock_config, tts=None, text="서울 지금 우산 필요해?",
dialogue_memory=dialogue_memory, history=HistoryContext(), legacy_memory=False,
intent_judge=True)
return reply, invoked, seen, planner.call_count
@pytest.mark.unit
@pytest.mark.parametrize("routed", [["getWeather", "stop"], ["logMeal", "stop"], ["screenView"]])
def test_a_lone_one_step_tool_needs_no_plan(mock_config, db, dialogue_memory, routed):
_, _, _, planned = _run(mock_config, db, dialogue_memory, routed=routed,
chat_replies=[_native_call(routed[0], {}), _content(GROUNDED)])
assert planned == 0
@pytest.mark.unit
@pytest.mark.parametrize("routed", [
["webSearch", "stop"], # searches chain (who directed it, what else they made)
["schedule", "stop"], # list, then cancel the one asked about
["getWeather", "webSearch", "stop"], # two tools
], ids=["search", "schedule", "two tools"])
def test_turns_that_may_take_several_steps_keep_the_plan(mock_config, db, dialogue_memory, routed):
_, _, _, planned = _run(mock_config, db, dialogue_memory, routed=routed,
chat_replies=[_content(GROUNDED)])
assert planned == 1
@pytest.mark.unit
def test_a_tool_less_answer_on_a_one_step_turn_is_dropped_and_the_tool_called(mock_config, db, dialogue_memory):
reply, invoked, seen, _ = _run(
mock_config, db, dialogue_memory, routed=["getWeather", "stop"],
chat_replies=[_content(FABRICATED), _native_call("getWeather", {"location": "Seoul"}),
_content(GROUNDED)])
assert invoked == ["getWeather"]
assert reply == GROUNDED
# The made-up answer never reaches a later turn, and the retry names the tool.
assert not any(FABRICATED in (m.get("content") or "") for m in seen[1])
assert "getWeather" in (seen[1][-1].get("content") or "")
@pytest.mark.unit
def test_a_second_made_up_answer_falls_back_to_the_plan_and_its_tool(mock_config, db, dialogue_memory):
# Asked once, the model still answers without the tool: the turn takes the planned path, the
# skipped-tool guard runs the planned call, and no made-up answer reaches the user.
reply, invoked, seen, planned = _run(
mock_config, db, dialogue_memory, routed=["getWeather", "stop"],
plan=["getWeather location='Seoul'", "Reply to the user with the weather."],
chat_replies=[_content(FABRICATED), _content(FABRICATED), _content(GROUNDED)])
assert planned == 1
assert invoked == ["getWeather"]
assert reply == GROUNDED
assert not any(FABRICATED in (m.get("content") or "") for m in seen[-1])
@pytest.mark.unit
def test_a_question_back_goes_out_when_the_plan_needs_no_tool(mock_config, db, dialogue_memory):
# The late plan judges that no tool is needed (a question back to the user): it is delivered.
question = "어느 도시 날씨를 알려 드릴까요?"
reply, invoked, seen, planned = _run(
mock_config, db, dialogue_memory, routed=["getWeather", "stop"],
chat_replies=[_content(question), _content(question)])
assert planned == 1
assert invoked == []
assert reply == question
assert len(seen) == 2
@pytest.mark.unit
def test_a_model_that_calls_the_tool_is_not_asked_again(mock_config, db, dialogue_memory):
reply, invoked, seen, _ = _run(
mock_config, db, dialogue_memory, routed=["getWeather", "stop"],
chat_replies=[_native_call("getWeather", {"location": "Seoul"}), _content(GROUNDED)])
assert invoked == ["getWeather"]
assert reply == GROUNDED
assert len(seen) == 2
@pytest.mark.unit
def test_v1_turns_still_plan_a_one_step_tool(mock_config, db, dialogue_memory):
from jarvis.reply import engine as engine_mod
mock_config.ollama_chat_model = NATIVE_MODEL
mock_config.evaluator_enabled = False
with patch.object(engine_mod, "chat_with_messages", return_value=_content(GROUNDED)), \
patch.object(engine_mod, "select_tools", return_value=["getWeather", "stop"]), \
patch.object(engine_mod, "extract_search_params_for_memory", return_value={"keywords": []}), \
patch.object(engine_mod, "plan_query", return_value=["Reply to the user."]) as planner:
engine_mod.run_reply_engine(db=db, cfg=mock_config, tts=None, text="서울 지금 우산 필요해?",
dialogue_memory=dialogue_memory)
assert planner.call_count == 1

View File

@@ -231,3 +231,41 @@ def test_dictation_clean_up_uses_chat_completions():
out = _llm_clean_dictation("um hello there", V1, model="qwen")
assert out == "hello there"
assert post.call_args.args[0] == f"{V1}/chat/completions"
# One-shot calls (docs/llm_contexts.md, transport note): a call no later request continues says so
# ("strata_checkpoint": false), so Strata gives its session away first and a reply's tool turn keeps
# its own session through the short calls made between its steps.
def test_short_calls_are_marked_one_shot():
from jarvis.listening.intent_judge import IntentJudge, IntentJudgeConfig, warm_up_ollama_model
from jarvis.listening.transcript_buffer import TranscriptSegment
answer = json.dumps({"directed": True, "query": "q", "stop": False, "confidence": "high", "reasoning": "r"})
seg = TranscriptSegment(text="jarvis q", start_time=1.0, end_time=2.0)
calls = {
"direct": lambda: call_llm_direct(V1, "qwen", "s", "u"),
"streaming": lambda: call_llm_streaming(V1, "qwen", "s", "u"),
"warm-up": lambda: warm_up_ollama_model(V1, "qwen", timeout=5),
"intent judge": lambda: IntentJudge(IntentJudgeConfig(ollama_base_url=V1, model="qwen")).judge([seg], wake_timestamp=1.0),
}
for name, call in calls.items():
resp = _sse({"choices": [{"delta": {"content": "x"}}]}) if name == "streaming" else _response(_completion(answer))
with patch("jarvis.llm.requests.post", return_value=resp) as post:
call()
assert post.call_args.kwargs["json"].get("strata_checkpoint") is False, name
def test_the_reply_loop_is_not_marked_one_shot():
# its next step continues this one, so its session must be kept
with patch("jarvis.llm.requests.post", return_value=_response(_completion("ok"))) as post:
chat_with_messages(V1, "qwen", [{"role": "user", "content": "hi"}],
tools=[{"type": "function", "function": {"name": "getWeather"}}])
assert "strata_checkpoint" not in post.call_args.kwargs["json"]
def test_ollama_api_gets_no_one_shot_field():
with patch("jarvis.llm.requests.post", return_value=_response({"message": {"content": "ok"}})) as post:
call_llm_direct("http://127.0.0.1:11434", "m", "s", "u")
assert "strata_checkpoint" not in post.call_args.kwargs["json"]