javis_bot/src/jarvis/reply/planner.spec.md

# Task-list planner

## Purpose

Small chat models (gemma4:e2b class) don't reliably decompose multi-step
queries turn-by-turn. They stop after one tool call when a second is
needed, echo the raw user utterance into tool arguments, or skip tools
entirely and confabulate from training. The planner fixes this by
running a single cheap classification-shaped LLM pass **at the very
front of the reply flow** that emits a short ordered list of sub-tasks.

The planner runs **after the tool router** and **before memory search**.
The router narrows the catalogue first so the planner's tool steps reference
concrete chosen names; the planner then **gates memory enrichment** and
**drives direct execution** for small models.

The engine uses the plan for three things:
1. **Gate memory enrichment** — the planner emits an explicit
   `searchMemory topic='<topic>'` directive on queries that need past
   user context; we skip the keyword-extraction LLM call, the diary
   / graph lookup, and the memory-digest LLM call otherwise.
2. **Confirm the tool allow-list** — the router's picks are
   authoritative; the tool names the planner references are unioned
   in as a safety net. Feeding the planner the narrowed catalogue
   (instead of the full 30+ list) stops small planners from
   paraphrasing ("get the weather") and from defaulting to
   `webSearch` when a more specific tool exists.
3. **Drive direct execution** for small models, as before — each
   planned step is resolved to a concrete tool call without
   round-tripping the chat model for intermediate turns.

## Scope

This spec covers `src/jarvis/reply/planner.py` and the engine
integration in `src/jarvis/reply/engine.py`.

## Behaviour

### When the planner runs

- After the dialogue context is assembled, MCP tools are loaded, and
  the tool router has produced a narrowed catalogue. Memory search
  runs *after* the planner so it can be gated on its output.
- The planner sees the **router-narrowed** tool catalogue (name +
  one-line description), not the full 30+ list. It does not see memory
  content — it decides whether memory is needed, via the
  `searchMemory` directive.
- Only when the query is at least `MIN_QUERY_CHARS` long (default 4).
  Pure noise like "hi" / "ok" still short-circuits.
- Only when `cfg.planner_enabled` is True (default).
- Only when an `ollama_base_url` and a resolvable model are available.

### Model resolution

1. `cfg.planner_model` (explicit override, for benchmarking)
2. `cfg.ollama_chat_model`

The planner must track the chat model. The plan is the scaffolding the
chat model follows; a weaker planner on top of a stronger chat model
produces bad scaffolding the chat model then fights against. The chat
model is also the one the user picked during setup as their quality
target, so upgrading it (through the setup wizard or config) must
automatically upgrade plan quality without requiring a second choice.

Note: the planner pays a cache miss relative to the tool router, which
*does* ride the warm small model. This is the intended trade-off —
plan quality drives everything downstream, router quality only narrows
one turn's allow-list.

### Prompt contract (plan_query)

The planner prompt instructs the model to emit:

- Short imperative sub-tasks, one per line.
- At most `MAX_STEPS` (default 5) steps.
- As the FIRST step, a `searchMemory topic='<topic>'` directive **only
  when** answering requires information the user shared in prior
  conversations. Omit otherwise — every extra directive is an
  avoidable LLM call downstream.
- Tool names from the provided catalog only (exact match), for any
  concrete tool step.
- Concrete arguments composed against dialogue context, not the raw
  utterance. Optional arguments that the user did not supply must be
  omitted, not fabricated from unrelated words.
- Angle-bracket placeholders (e.g. `<director name from step 1>`) for
  entities the lookup will reveal at runtime.
- Pronouns and demonstratives in the user query ("he", "his", "her",
  "their", "it", "that film") must be resolved against the dialogue
  context before emitting the step. Tools never see prior turns, so
  the named entity has to appear literally inside the tool argument
  string — `webSearch query='Harry Styles most famous songs'`, not
  `webSearch query='his most famous songs'`.
- A final synthesis/reply step when any `searchMemory` or tool step
  was planned.
- Steps in the same language the user wrote the query in.

### Parsing and hygiene

- Numbering (`1.`, `1)`), bullets (`-`, `*`, `•`), wrapping quotes,
  and markdown fences are stripped.
- Overlong steps (>200 chars) are truncated with an ellipsis.
- The list is capped at `MAX_STEPS`.
- The planner no longer filters out 1-step plans. A single
  `["Reply to the user."]` plan is the planner's *positive* decision
  that no memory or tools are needed — the engine uses that to skip
  the memory extractor, the tool router, and the direct-exec path
  entirely. Only an **empty** list means "planner failed / disabled;
  fall open to legacy safe defaults" (run memory enrichment + tool
  router). The two states must stay distinguishable.

### Engine integration

The engine consumes the plan in two phases.

**Phase 1 — preparation gating (before the turn loop starts):**

- `plan_requires_memory(plan)` — true iff any step is a `searchMemory`
  directive. The engine uses it to gate the entire memory-enrichment
  block (keyword extractor LLM call, diary / graph lookups, digest
  LLM call). Optional `memory_topic_of(step)` extracts the directive's
  `topic='...'` hint, threaded into the keyword extractor so it
  anchors on what the planner wanted to look up rather than
  re-deriving from the raw utterance.
- `tool_names_in_plan(plan, known_names)` — ordered de-duped list of
  tool names the planner referenced. The engine unions this into the
  router-selected allow-list (never replaces it). `stop` and
  `toolSearchTool` are always added regardless.
- `plan_has_unresolved_tool_steps(plan, known_names)` — true when the
  plan has non-synthesis steps but names no known tool (e.g. the
  model wrote `get the weather` instead of `getWeather ...`). In
  this state the direct-exec path is skipped — vague step text
  would otherwise force the resolver LLM to guess arguments (e.g.
  emitting `location='Nowhere'` for a bare weather request). The
  chat model takes the turn instead, using the router-selected
  allow-list.
- `strip_memory_directives(plan)` — the engine strips the
  `searchMemory` step from the plan once memory has been fetched, so
  downstream consumers (system-message injection, direct-exec,
  progress nudge) see a plan of pure tool + synthesis steps.

**Phase 2 — loop integration (existing behaviour):**

- `format_plan_block(steps)` renders an `ACTION PLAN:` block that is
  appended to the initial system message. Empty plan renders nothing.
  Single-step reply-only plans are not rendered either — they are
  noise to the chat model since the plan just says "reply".
- `progress_nudge(steps, tool_results_so_far)` produces a remainder
  hint injected after each tool result, naming the next planned step
  and reminding the model to substitute discovered entities and avoid
  duplicate arguments.
- When `use_text_tools` is active and the plan still has unexecuted
  tool steps, the engine runs `resolve_next_tool_call` to convert the
  next step into a concrete `{name, arguments}` JSON and dispatches
  the tool directly, bypassing the chat model for that turn. This
  keeps small models on-rails without relying on their native
  tool-call reliability.
- The chat model still runs the final synthesis turn so the reply is
  phrased in the daemon's voice using its own profile and persona.

### resolve_next_tool_call

- **Fast path**: if the step text is fully concrete (tool name in the
  allow-list + `key='value'` / `key="value"` pairs matching the tool's
  declared property keys, and no `<placeholder>`), parse it
  deterministically and return without any LLM call. This removes the
  resolver LLM as a failure surface for the common case — small models
  occasionally flake (timeout, empty, spurious `null`) even on
  trivially-concrete steps like `webSearch query='foo'`, which used to
  fall back to the chat model and produce a refusal instead of the
  search. The fast path is purely regex-driven, language-agnostic, and
  never calls the model.
- **LLM path**: when the step contains a `<placeholder>`, uses unknown
  argument keys, or doesn't fit the `key=value` shape, the step is
  passed to the LLM resolver which can substitute entities from prior
  results and remap names.
- Returns `None` for synthesis steps (the LLM emits the literal
  `null`), unknown tools, or invalid JSON. All `None` paths fall back
  to the normal chat-model turn.
- Validates the tool name against the provided schema's allow-list.
- Filters the returned `arguments` against the tool's declared
  JSON-schema property keys; unknown keys are dropped before dispatch.
  Tools that declare no properties keep the args as-is (they are
  free-form by design).
- Tolerates markdown fences the model may add despite instructions.
- Both planner LLM calls (`plan_query` and `resolve_next_tool_call`)
  request `num_ctx=8192` from Ollama so enriched memory and tool
  catalogue don't silently truncate in the 4096-token default window.

## Fail-open invariants

- Timeout, empty response, or exception in the planner LLM call →
  return `[]`.
- Invalid JSON in the step resolver → return `None` and let the chat
  model handle the turn normally.
- No plan never worsens the baseline; the engine behaves exactly as it
  did pre-planner.

## Configuration

| Key | Default | Purpose |
|-----|---------|---------|
| `planner_enabled` | `True` | Feature gate. |
| `planner_model` | `""` | Explicit planner model override. |
| `planner_timeout_sec` | `6.0` | Timeout for plan and step-resolver LLM calls. |

## Non-goals

- The planner does not re-plan mid-turn. If the emitted plan is wrong,
  the engine still progresses via the chat model's native tool calls.
  When the chat model produces natural-language content the loop
  terminates immediately.
- The planner does not validate semantic correctness of the plan; it
  trusts the model to produce sensible steps and relies on the
  resolver's schema-level guard to reject unknown tools.
- Plans are not cached across turns. Each user utterance gets its own
  plan because the dialogue state and entity references change.