fix(reply): a one-step turn whose model skips the tool twice is planned and guarded
After the one nudge, a second answer without the tool no longer goes out as it is: the turn takes the planned path (the planner runs then, one extra call only on this failure) and the skipped-tool guard runs the planned tool, so a made-up "24 degrees" answer cannot reach the user. A plan that needs no tool (a question back to the user) still lets the answer go out. Evals (qwen3:8b, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1, 3 runs): the five tool-skip/weather/screen/follow-up/router files 72/84 (71/84 on 0.1.20, same known failures); Possessor director-then-filmography 0/3, also 0/3 on 0.1.20 (qwen3:8b stops after one search; webSearch turns are still planned). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -161,7 +161,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
|
||||
## 12. Task-list Planner (pre-flight decomposition, gates the whole turn)
|
||||
|
||||
- **File**: [src/jarvis/reply/planner.py](src/jarvis/reply/planner.py) — `plan_query()`.
|
||||
- **Trigger**: once per reply, **after the tool router and before memory search**. **v2 brain** (`intent_judge=True`): runs only on turns where the router picked a tool (not for the sentinels `stop` or `toolSearchTool` alone; a router-picked `chatChannels` counts), so a turn without tools has one LLM call before its answer. Also skipped when the router's only pick is a one-step tool (`_ONE_STEP_TOOLS` in `engine.py`: `getWeather`, `logMeal`, `fetchMeals`, `screenView`) and the chat model calls tools natively; the chat model then gets the tool directly, and an answer it gives without the tool is dropped and followed by one nudge to call it (planner.spec.md, "Intent judge (v2)"). Skipped when `cfg.planner_enabled = False`, when the query is shorter than `MIN_QUERY_CHARS` (4), or when no model / base URL is available.
|
||||
- **Trigger**: once per reply, **after the tool router and before memory search**. **v2 brain** (`intent_judge=True`): runs only on turns where the router picked a tool (not for the sentinels `stop` or `toolSearchTool` alone; a router-picked `chatChannels` counts), so a turn without tools has one LLM call before its answer. Also skipped when the router's only pick is a one-step tool (`_ONE_STEP_TOOLS` in `engine.py`: `getWeather`, `logMeal`, `fetchMeals`, `screenView`) and the chat model calls tools natively; the chat model then gets the tool directly, and an answer it gives without the tool is dropped and followed by one nudge to call it; a second answer without the tool runs the planner after all (one extra call, only then) and the skipped-tool guard applies (planner.spec.md, "Intent judge (v2)"). Skipped when `cfg.planner_enabled = False`, when the query is shorter than `MIN_QUERY_CHARS` (4), or when no model / base URL is available.
|
||||
- **Model / gating**: resolution chain `planner_model (override) → ollama_chat_model`. The planner tracks the chat model so upgrading the chat model (via setup wizard or config) automatically upgrades plan quality.
|
||||
- **Inputs**: user query, dialogue context, the live time/location line (`LIVE CONTEXT:`, so a location-dependent ask plans `searchMemory topic='where the user lives'` only when the location is unknown or disabled), **router-narrowed** tool catalogue (names + one-line descriptions + argument names, built by `planner_tool_catalog()`) — not the full 30+ list. When the carry-over guard from #7 fires, the previous turn's failed tool name is unioned into this catalogue before the planner sees it, so the planner can plan a re-call without `toolSearchTool` round-tripping. **No** memory context — the planner decides *whether* memory is needed.
|
||||
- **System prompt**: `_PROMPT_TEMPLATE` in `planner.py`. Teaches the `searchMemory topic='...'` directive for prior-conversation lookups, short imperative tool steps, angle-bracket entity placeholders, final synthesis step, same-language output (the words of search and play arguments too, repeated in the user message's closing line), no numbering.
|
||||
@@ -215,7 +215,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
|
||||
| 10 | Graph extraction | ~1/session | No (background) | LARGE |
|
||||
| 11 | Graph best-child | 0-N | No (background) | SMALL (via router chain) |
|
||||
| 11b | Graph node merge | 0-N (per node, batched) | No (background) | SMALL (via router chain) |
|
||||
| 12 | Planner (plan_query) | 0-1 (v2: tool turns except one-step tools) | yes (planner_enabled) | LARGE/SMALL (tracks chat model) |
|
||||
| 12 | Planner (plan_query) | 0-1 (v2: tool turns except one-step tools, which plan only when the model skips the tool twice) | yes (planner_enabled) | LARGE/SMALL (tracks chat model) |
|
||||
| 13 | Plan step resolver | 0-N (SMALL only) | auto by size + plan | SMALL (via router chain) |
|
||||
| 14 | Tool-specific | per-tool | n/a | LARGE |
|
||||
| 15 | Voice call judgements | 0 (per utterance / tie, outside the reply) | call mode / policy | LARGE (chat model) |
|
||||
|
||||
Reference in New Issue
Block a user