diff --git a/docs/llm_contexts.md b/docs/llm_contexts.md index 2dcc716..163bb94 100644 --- a/docs/llm_contexts.md +++ b/docs/llm_contexts.md @@ -164,7 +164,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i - **Trigger**: once per reply, **after the tool router and before memory search**. **v2 brain** (`intent_judge=True`): runs only on turns where the router picked a tool (not for the sentinels `stop` or `toolSearchTool` alone; a router-picked `chatChannels` counts), so a turn without tools has one LLM call before its answer. Also skipped when the router's only pick is a one-step tool (`_ONE_STEP_TOOLS` in `engine.py`: `getWeather`, `logMeal`, `fetchMeals`, `screenView`) and the chat model calls tools natively; the chat model then gets the tool directly, and an answer it gives without the tool is dropped and followed by one nudge to call it; a second answer without the tool runs the planner after all (one extra call, only then) and the skipped-tool guard applies (planner.spec.md, "Intent judge (v2)"). Skipped when `cfg.planner_enabled = False`, when the query is shorter than `MIN_QUERY_CHARS` (4), or when no model / base URL is available. - **Model / gating**: resolution chain `planner_model (override) → ollama_chat_model`. The planner tracks the chat model so upgrading the chat model (via setup wizard or config) automatically upgrades plan quality. - **Inputs**: user query, dialogue context, the live time/location line (`LIVE CONTEXT:`, so a location-dependent ask plans `searchMemory topic='where the user lives'` only when the location is unknown or disabled), **router-narrowed** tool catalogue (names + one-line descriptions + argument names, built by `planner_tool_catalog()`) — not the full 30+ list. When the carry-over guard from #7 fires, the previous turn's failed tool name is unioned into this catalogue before the planner sees it, so the planner can plan a re-call without `toolSearchTool` round-tripping. **No** memory context — the planner decides *whether* memory is needed. -- **System prompt**: `_PROMPT_TEMPLATE` in `planner.py`. Teaches the `searchMemory topic='...'` directive for prior-conversation lookups, short imperative tool steps, angle-bracket entity placeholders, final synthesis step, same-language output (the words of search and play arguments too, repeated in the user message's closing line), no numbering. +- **System prompt**: `_PROMPT_TEMPLATE` in `planner.py`. Teaches the `searchMemory topic='...'` directive for prior-conversation lookups, short imperative tool steps, angle-bracket entity placeholders, final synthesis step, same-language output in the query's own writing system (the words of search and play arguments too, repeated in the user message's closing line), no numbering. - **Output**: list of plan steps (max `MAX_STEPS` = 5). Gates memory enrichment (#3 / #4) and augments the tool router (#7 — planner's picks are unioned in, not replacing). Single-step `["Reply to the user."]` plans are the planner's positive "no memory, no tools" signal. An empty list is fail-open — the engine reverts to running #3 unconditionally. Consumed further by the engine to build the `ACTION PLAN:` system-message block and drive the direct-exec loop (#13) for small models. - **Limits**: `planner_timeout_sec` (6s). Fail-open → `[]`. diff --git a/evals/test_search_in_users_language.py b/evals/test_search_in_users_language.py index 2d63acf..f4e5852 100644 --- a/evals/test_search_in_users_language.py +++ b/evals/test_search_in_users_language.py @@ -22,6 +22,8 @@ CASES = [ ("netflix", "요즘 볼 만한 넷플릭스 드라마 세 개만 추천해 줘"), ("price", "엔비디아 RTX 5050 그래픽카드 가격 검색해 줘"), ("restaurants", "이번 주말에 갈 만한 맛집 좀 찾아 줘"), + # Live 2026-11-18: the search was "사과와 배 중哪个가 더 단가요" (another script mixed in). + ("fruit", "자비 사과랑 배 중에 뭐가 더 달아 검색해줘"), ] diff --git a/src/jarvis/reply/planner.py b/src/jarvis/reply/planner.py index b71568c..e5ea417 100644 --- a/src/jarvis/reply/planner.py +++ b/src/jarvis/reply/planner.py @@ -235,7 +235,9 @@ _PROMPT_TEMPLATE = ( "12. Write the steps in the same language the user wrote the query in. " "The words of every search or play argument stay in the user's " "language too, never translated into English, because a search in the " - "user's language finds what people where they are get.\n" + "user's language finds what people where they are get. Use only the " + "writing system of the query; never mix in characters of another " + "script.\n" ) @@ -283,7 +285,7 @@ def _build_user_message( parts.append("DIALOGUE CONTEXT: (empty)") parts.append(f"USER QUERY: {query.strip()}") parts.append("\nEmit the plan now, one step per line, no numbering, written in the language of " - "the USER QUERY (the words of searches too).") + "the USER QUERY (the words of searches too), in its own letters only.") return "\n\n".join(parts) diff --git a/src/jarvis/reply/planner.spec.md b/src/jarvis/reply/planner.spec.md index e96498f..acf0e67 100644 --- a/src/jarvis/reply/planner.spec.md +++ b/src/jarvis/reply/planner.spec.md @@ -116,6 +116,13 @@ The planner prompt instructs the model to emit: (qwen3:8b, 3 runs of 3 Korean requests): Korean searches 6/9 before ("요즘 볼 만한 넷플릭스 드라마" searched "best netflix dramas 2026" 3/3), 8/9 after (that one 2/3). +- Only the writing system of the query, no characters of another script, + in the rule and in the closing line ("in its own letters only"). Live + 2026-11-18 "자비 사과랑 배 중에 뭐가 더 달아" was searched as + "사과와 배 중哪个가 더 단가요". Same eval, case `fruit` (qwen3.6-35b-a3b): + Han in the search 7/7 before; with only the rule 3/3, with only the + closing line 3/3, with both 0/6, the other cases unchanged (12/12). + `webSearch` also drops such letters itself (web_search.spec.md). ### Parsing and hygiene diff --git a/src/jarvis/tools/builtin/web_search.spec.md b/src/jarvis/tools/builtin/web_search.spec.md index d6332d0..9ae1c74 100644 --- a/src/jarvis/tools/builtin/web_search.spec.md +++ b/src/jarvis/tools/builtin/web_search.spec.md @@ -17,8 +17,8 @@ Netflix dramas searched in English and recommended American shows; eval Letters of a writing system the user did not use are dropped from the query before any provider runs (`_drop_foreign_script`, by Unicode script name against `context.redacted_text`; Latin is always kept for brand names and model numbers). Live 2026-11-18 a Hangul question was searched as -"사과와 배 중哪个가 더 단가요". The same eval checks that no other script is mixed in; the planner -alone did not reproduce it in 24 runs, so no prompt rule was added for it. +"사과와 배 중哪个가 더 단가요". The planner is also told to use only the query's writing system +(planner.spec.md); `evals/test_search_in_users_language.py` checks that no other script is mixed in. ### Real-time info routing (`STREAM_BROWSER`)