fix(reply): known user location reaches getWeather; retries are not blocked by the previous reply

- Drop the engine's deterministic Korean weather shortcut. It bypassed the
  chat model and the tool runner, so a location the user had stated could
  never be used. The tool already returns one clean Korean sentence.
- getWeather and the tool-argument prompts pass the place the user has
  said they are (conversation or memory) and leave the argument empty
  otherwise, so the detected location is not copied into it.
- getWeather retries a "district, city, country" name with its leading
  parts dropped when the geocoder finds nothing.
- The planner gets the live time/location line and plans
  searchMemory topic='where the user lives' only when the location is
  unknown or disabled.
- The per-reply repeat guard counts only this reply's tool messages, so
  calls carried over from the previous reply no longer block a retry the
  user asks for.
- Memory extractor: the context block sits after the examples and the
  no-location example uses a different query, so a location already in
  context is not asked for again. New counterpart eval checks that it is
  still asked for when the context has none.
- Tool retry eval fails every call in turn 1, so a model that retries by
  itself inside turn 1 still exercises the user's "try again".
- Specs (planner, reply), docs/llm_contexts.md and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-11-11 21:55:45 +09:00
parent 4bb3895ca0
commit 3ec430fd25
15 changed files with 255 additions and 79 deletions

View File

@@ -46,7 +46,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
- **File**: [src/jarvis/reply/enrichment.py](src/jarvis/reply/enrichment.py) — `extract_search_params_for_memory()` (~line 71).
- **Trigger**: once per reply, **only when the pre-flight planner (#12) emitted a `searchMemory` directive or returned an empty plan (fail-open)**. Pure reply-only plans skip this entirely — saves one LLM call per greeting / small-talk turn.
- **Model / gating**: resolved via `resolve_tool_router_model(cfg)` — `tool_router_model → intent_judge_model → ollama_chat_model`. Small classification task; rides the same small/warm model as the router. Silent empty-dict on failure.
- **Inputs**: user query (with the planner's `topic` hint appended when present), optional context hint (live-context compact summary), UTC now.
- **Inputs**: user query (with the planner's `topic` hint appended when present), optional context hint (live-context compact summary, placed after the examples so the model checks its questions against it last), UTC now.
- **System prompt**: inline at [enrichment.py:35-63](src/jarvis/reply/enrichment.py:35).
- **Output**: `{keywords, from?, to?, questions?}`. Consumed by memory search in the reply engine.
- **Limits**: up to 2 retries; timeout from `llm_tools_timeout_sec`.
@@ -158,7 +158,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
- **File**: [src/jarvis/reply/planner.py](src/jarvis/reply/planner.py) — `plan_query()`.
- **Trigger**: once per reply, **after the tool router and before memory search**. Skipped when `cfg.planner_enabled = False`, when the query is shorter than `MIN_QUERY_CHARS` (4), or when no model / base URL is available.
- **Model / gating**: resolution chain `planner_model (override) → ollama_chat_model`. The planner tracks the chat model so upgrading the chat model (via setup wizard or config) automatically upgrades plan quality.
- **Inputs**: user query, dialogue context, **router-narrowed** tool catalogue (names + one-line descriptions + argument names, built by `planner_tool_catalog()`) — not the full 30+ list. When the carry-over guard from #7 fires, the previous turn's failed tool name is unioned into this catalogue before the planner sees it, so the planner can plan a re-call without `toolSearchTool` round-tripping. **No** memory context — the planner decides *whether* memory is needed.
- **Inputs**: user query, dialogue context, the live time/location line (`LIVE CONTEXT:`, so a location-dependent ask plans `searchMemory topic='where the user lives'` only when the location is unknown or disabled), **router-narrowed** tool catalogue (names + one-line descriptions + argument names, built by `planner_tool_catalog()`) — not the full 30+ list. When the carry-over guard from #7 fires, the previous turn's failed tool name is unioned into this catalogue before the planner sees it, so the planner can plan a re-call without `toolSearchTool` round-tripping. **No** memory context — the planner decides *whether* memory is needed.
- **System prompt**: `_PROMPT_TEMPLATE` in `planner.py`. Teaches the `searchMemory topic='...'` directive for prior-conversation lookups, short imperative tool steps, angle-bracket entity placeholders, final synthesis step, same-language output, no numbering.
- **Output**: list of plan steps (max `MAX_STEPS` = 5). Gates memory enrichment (#3 / #4) and augments the tool router (#7 — planner's picks are unioned in, not replacing). Single-step `["Reply to the user."]` plans are the planner's positive "no memory, no tools" signal. An empty list is fail-open — the engine reverts to running #3 unconditionally. Consumed further by the engine to build the `ACTION PLAN:` system-message block and drive the direct-exec loop (#13) for small models.
- **Limits**: `planner_timeout_sec` (6s). Fail-open → `[]`.

File diff suppressed because one or more lines are too long

View File

@@ -500,6 +500,39 @@ class TestMemoryEnrichment:
assert not any("locat" in q or "where" in q for q in questions), \
f"Should not ask about location when it's in context. Got: {questions}"
@pytest.mark.eval
@requires_judge_llm
def test_enrichment_asks_location_when_context_lacks_it(self, mock_config):
"""
Counterpart of the test above: when the live context has no location,
a query that depends on where the user is still asks for it, so the
memory lookup can find where the user said they live.
"""
from jarvis.reply.enrichment import extract_search_params_for_memory
from helpers import JUDGE_MODEL
mock_config.ollama_base_url = JUDGE_BASE_URL
mock_config.ollama_chat_model = JUDGE_MODEL
result = extract_search_params_for_memory(
query="recommend a restaurant I'd enjoy",
ollama_base_url=mock_config.ollama_base_url,
ollama_chat_model=mock_config.ollama_chat_model,
timeout_sec=15.0,
context_hint=(
"Current local time: Sunday, 2026-04-19 14:30 local. "
"Location: Unknown"
),
)
questions = [q.lower() for q in result.get("questions", [])]
print(f"\n📊 Questions without a location in context: {questions}")
assert result.get("keywords"), \
f"Extractor returned no keywords. Result: {result}"
assert any("locat" in q or "where" in q for q in questions), \
f"Should ask where the user is when the context has no location. Got: {questions}"
@pytest.mark.eval
def test_enrichment_provides_context_to_llm(self, mock_config, eval_db, eval_dialogue_memory):
"""
@@ -1030,24 +1063,27 @@ class TestHelpfulness:
is_small = detect_model_size(JUDGE_MODEL) == ModelSize.SMALL
call_count = {"n": 0}
# Every call in turn 1 fails and every call in turn 2 succeeds. A
# model that retries by itself inside turn 1 must not get the data
# there, or turn 2 is answered from that result and the user's
# "try again" is never tested.
phase = {"turn": 1}
def mock_tool_run(db, cfg, tool_name, tool_args, **kwargs):
"""First call returns error, second call succeeds."""
"""Fails throughout turn 1, succeeds in turn 2."""
from jarvis.tools.types import ToolExecutionResult
capture.record(tool_name, tool_args or {})
call_count["n"] += 1
if tool_name == "getWeather":
if call_count["n"] <= 1:
# First call: tool can't find the location
if phase["turn"] == 1:
# Turn 1: tool can't find the location
return ToolExecutionResult(
success=False,
reply_text="",
error_message="Could not find location 'Kazbegi'. Try a different spelling or a nearby city."
)
else:
# Subsequent calls: tool succeeds
# Turn 2: tool succeeds
return ToolExecutionResult(
success=True,
reply_text="Current weather near Kazbegi (Stepantsminda), Georgia:\nConditions: Partly cloudy\nTemperature: 2.5°C\nWind: 25 km/h\n7-day: 2026-04-10: -1–5°C, Snow showers"
@@ -1070,6 +1106,7 @@ class TestHelpfulness:
# Turn 2: User insists on retry — tool should succeed this time
capture.clear()
phase["turn"] = 2
response = run_reply_engine(
db=eval_db, cfg=mock_config, tts=None,
text=follow_up,

View File

@@ -16,8 +16,8 @@ answers without it. This eval drives the real engine (planner, router and
chat model are live; only the tools are mocked) and checks both shapes:
- live facts: an umbrella / rain question must reach getWeather. The
queries avoid the words the deterministic weather shortcut matches, so
the chat model has to choose the tool itself.
queries avoid the word "weather", so the chat model has to see that
the question needs the tool.
- actions: "log my meal" must reach logMeal, and a timer request (no timer
tool exists) must not be answered as if a timer had been set.

View File

@@ -940,49 +940,6 @@ def _maybe_deterministic_site_search(text: str, db, cfg, language) -> Optional[s
return None
_WEATHER_INTENT_WORDS = (
"날씨", "기온", "더워", "더운", "추워", "추운", "비 와", "비와", "비 올",
"눈 와", "눈와", "weather", "temperature", "forecast",
)
def _maybe_deterministic_weather(text: str, db, cfg, language) -> Optional[str]:
"""Run getWeather directly and return its concise Korean sentence, bypassing
the chat model. The 7B otherwise re-synthesises the weather into multiple
sentences and leaks units ("25도 Celsius"); the tool already formats one
clean Korean sentence, so for a plain weather ask we just return it.
Fail-open: any problem returns None and the normal flow continues.
"""
try:
low = (text or "").lower()
if not any(w in low for w in _WEATHER_INTENT_WORDS):
return None
# Extract a city candidate from the utterance (GeoIP auto-detect is
# unavailable in the container, so a named city must be passed through).
import re
_loc = text
for w in _WEATHER_INTENT_WORDS + (
"알려줘", "어때", "어떄", "말해줘", "확인해줘", "확인", "해줘",
"오늘", "지금", "현재", "좀", "그래서", "그럼",
):
_loc = re.sub(re.escape(w), " ", _loc, flags=re.IGNORECASE)
_loc = re.sub(r"(은|는|이|가|의|에|에서|로|을|를)\b", " ", _loc)
_loc = re.sub(r"\s+", " ", _loc).strip(" .,!?。")
args = {"location": _loc} if 1 <= len(_loc) <= 12 else {}
from ..tools.registry import run_tool_with_retries
res = run_tool_with_retries(
db=db, cfg=cfg, tool_name="getWeather", tool_args=args,
system_prompt="", original_prompt="", redacted_text=redact(text),
max_retries=1, language=language,
)
if res and getattr(res, "success", False) and res.reply_text:
debug_log("deterministic weather executed", "tools")
return res.reply_text
except Exception as e: # noqa: BLE001
debug_log(f"deterministic weather failed (fail-open): {e}", "tools")
return None
_HISTORY_FENCE = "conversation_records"
@@ -1061,13 +1018,6 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
if _site_search_reply is not None:
return _site_search_reply
# Step 0.6: Deterministic weather — return getWeather's concise Korean
# sentence directly so the chat model can't rephrase it into multiple
# sentences or leak units.
_weather_reply = _maybe_deterministic_weather(text, db, cfg, language)
if _weather_reply is not None:
return _weather_reply
# Step 2: Check for recent dialogue context
recent_messages = []
is_new_conversation = True
@@ -1312,6 +1262,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
query=redacted,
dialogue_context=_dialogue_ctx,
tools=_planner_tool_catalog,
live_context=_live_time_location_string(cfg),
)
except Exception as _plan_exc: # pragma: no cover — defensive
debug_log(f"planner step failed (non-fatal): {_plan_exc}", "planning")
@@ -2566,8 +2517,10 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# In native-tools mode results carry role="tool"; in text-tools mode they carry
# role="user" with a "tool_name" key — check both to make the guard effective
# in small-model paths where direct-exec is most likely to loop.
# Only this reply's messages count: results carried over from the
# previous reply must not block a retry the user just asked for.
duplicate_tool_count = sum(
1 for msg in messages[-10:]
1 for msg in messages[max(user_msg_index + 1, len(messages) - 10):]
if msg.get("tool_name") == tool_name
and msg.get("role") in ("tool", "user")
)

View File

@@ -37,9 +37,7 @@ def extract_search_params_for_memory(query: str, ollama_base_url: str, ollama_ch
Extract:
1. CONTENT KEYWORDS: 3-5 relevant topics/subjects (ignore time words). Include general, high-level category tags that would be suitable for blog-style tagging when applicable (e.g., "cooking", "fitness", "travel", "finance").
2. TIME RANGE: If mentioned, convert to exact timestamps
3. QUESTIONS: What implicit personal questions does this query need answered from stored knowledge about the user? These are things the assistant would need to know about the user to give a personalised answer. Omit if the query needs no personal context, OR if the answer is already visible in the ALREADY IN CONTEXT block below.
{hint_block}
3. QUESTIONS: What implicit personal questions does this query need answered from stored knowledge about the user? These are things the assistant would need to know about the user to give a personalised answer. Omit if the query needs no personal context, OR if the answer is already visible in the ALREADY IN CONTEXT block at the end.
Respond ONLY with JSON in this format:
{{"keywords": ["keyword1", "keyword2"], "questions": ["what are the user's food preferences?"], "from": "2025-08-21T00:00:00Z", "to": "2025-08-21T23:59:59Z"}}
@@ -57,10 +55,14 @@ Examples:
"remember that password I mentioned today?" → {{"keywords": ["password", "accounts", "security", "credentials"], "from": "2025-08-22T00:00:00Z", "to": "2025-08-22T23:59:59Z"}}
"what news might interest me?" → {{"keywords": ["interests", "hobbies", "preferences", "likes", "passionate"], "questions": ["what topics interest the user?", "what are the user's hobbies?"]}}
"news of interest to me" / "news that would interest me" / "news interesting for me" / "recall my interests and search for news on them" → {{"keywords": ["interests", "hobbies", "preferences", "likes", "passionate"], "questions": ["what topics interest the user?", "what are the user's hobbies?"]}}
"recommend a restaurant I'd enjoy" (no location in context) → {{"keywords": ["food preferences", "restaurants", "cuisine", "dining", "favorites"], "questions": ["what cuisine does the user like?", "where is the user located?"]}}
"recommend a restaurant I'd enjoy" (location already in context) → {{"keywords": ["food preferences", "restaurants", "cuisine", "dining", "favorites"], "questions": ["what cuisine does the user like?"]}}
"recommend a restaurant I'd enjoy" (ALREADY IN CONTEXT shows a location) → {{"keywords": ["food preferences", "restaurants", "cuisine", "dining", "favorites"], "questions": ["what cuisine does the user like?"]}}
"find me a gym I'd like" (ALREADY IN CONTEXT shows no location, or Location: Unknown) → {{"keywords": ["fitness", "gym", "exercise", "preferences"], "questions": ["what kind of exercise does the user enjoy?", "where is the user located?"]}}
"suggest a movie for me" → {{"keywords": ["movies", "films", "entertainment", "preferences", "genres"], "questions": ["what film genres does the user enjoy?", "what movies has the user watched recently?"]}}
"what time is it?" → {{"keywords": []}}
{hint_block}
Before answering, drop every question whose answer is shown in the block above.
"""
formatted_prompt = system_prompt.format(hint_block=hint_block)

View File

@@ -180,10 +180,17 @@ _PROMPT_TEMPLATE = (
"preferences and dietary restrictions'; 'suggest something I'd "
"enjoy watching' → searchMemory topic='user viewing tastes'. "
"NO-examples (DO NOT emit searchMemory): 'who is Britney Spears', "
"'what is the capital of France', 'what's the weather today', "
"'what is the capital of France', 'what's the weather today' "
"(when LIVE CONTEXT gives the location), "
"'search the web for Possessor 2020'. If no prior-conversation "
"memory is needed, OMIT this step entirely — every extra "
"searchMemory directive costs a real LLM call.\n"
"2b. A tool step that depends on a fact about the user themself "
"(where they are, for weather or local results) also needs memory "
"FIRST when neither LIVE CONTEXT nor DIALOGUE CONTEXT gives that "
"fact, e.g. LIVE CONTEXT says 'Location: Unknown' or 'Location: "
"Disabled': emit `searchMemory topic='where the user lives'` before "
"the tool step. When LIVE CONTEXT gives a location, do not.\n"
"3. Use external tools ONLY from the AVAILABLE TOOLS list below, "
"by exact name. If no tool is needed (greeting, small-talk, "
"opinion, a question about yourself, a fact already in the "
@@ -257,6 +264,7 @@ def _build_user_message(
query: str,
dialogue_context: str,
tools: Sequence[Tuple[str, str]],
live_context: str = "",
) -> str:
parts = []
if tools:
@@ -264,6 +272,8 @@ def _build_user_message(
parts.append(f"AVAILABLE TOOLS:\n{tool_lines}")
else:
parts.append("AVAILABLE TOOLS: (none — plan a direct reply)")
if live_context.strip():
parts.append(f"LIVE CONTEXT: {live_context.strip()}")
if dialogue_context.strip():
parts.append(f"DIALOGUE CONTEXT (most recent last):\n{dialogue_context.strip()}")
else:
@@ -445,6 +455,7 @@ def plan_query(
*,
timeout_sec: Optional[float] = None,
memory_context: str = "", # deprecated; planner now runs before memory
live_context: str = "",
) -> List[str]:
"""Run a short planning LLM pass over the query + dialogue context.
@@ -454,6 +465,10 @@ def plan_query(
A single ``["Reply to the user."]`` is a valid plan and means
"answer directly; skip both memory and tools".
``live_context`` is the engine's live time/location line. It lets the
planner tell a known location from an unknown one, and plan a memory
lookup for where the user is only in the second case.
``memory_context`` is accepted for backward compatibility with old
callers but no longer used: the planner runs before memory search
so it decides *whether* memory is needed, via the searchMemory
@@ -478,7 +493,7 @@ def plan_query(
)
system_prompt = _PROMPT_TEMPLATE.format(max_steps=MAX_STEPS)
user_content = _build_user_message(query, dialogue_context, tools)
user_content = _build_user_message(query, dialogue_context, tools, live_context)
try:
raw = call_llm_direct(

View File

@@ -69,6 +69,11 @@ one turn's allow-list.
### Prompt contract (plan_query)
The planner's user message also carries a `LIVE CONTEXT:` line: the
engine's live time/location string (`_live_time_location_string`), passed
as `plan_query(..., live_context=...)`. It lets the planner tell a known
location from an unknown one without seeing any memory content.
The planner's AVAILABLE TOOLS lines come from `planner_tool_catalog(schema)`:
the tool's first description line plus its argument names, e.g.
`getWeather: Weather only ... (args: location)`. Without the argument names
@@ -83,6 +88,11 @@ The planner prompt instructs the model to emit:
when** answering requires information the user shared in prior
conversations. Omit otherwise — every extra directive is an
avoidable LLM call downstream.
- The same directive (`searchMemory topic='where the user lives'`) before a
tool step that needs a fact about the user themself, such as where they
are, when neither the LIVE CONTEXT nor the dialogue gives it (the live
context says `Location: Unknown` or `Location: Disabled`). When the live
context gives a location, a plain weather ask stays memory-free.
- Tool names from the provided catalog only (exact match), for any
concrete tool step.
- Concrete arguments composed against dialogue context, not the raw

View File

@@ -119,10 +119,13 @@ TOOL_CONSTRAINTS_LARGE = (
"Tools may state in their description that an argument has a sensible default "
"(for example getWeather uses the user's current location when none is given). "
"Call the tool in the SAME turn with whatever arguments you have — even zero — "
"and let it fill the rest. Do NOT reply with a clarifying question like \"which "
"location?\" for an argument the tool auto-derives. Concretely: \"how's the "
"weather today\" must trigger getWeather immediately with no arguments, not a "
"question back to the user.\n\n"
"and let it fill the rest. What the user has told you counts as having it: if the "
"conversation or what you remember about the user gives the value, pass it "
"(\"how's the weather\" from a user who told you they live in Leeds calls "
"getWeather with location Leeds). Do NOT reply with a clarifying question like "
"\"which location?\" for an argument the tool auto-derives. Concretely: \"how's "
"the weather today\" with no place known must trigger getWeather immediately "
"with no arguments, not a question back to the user.\n\n"
"SELF-CONTAINED TOOL ARGUMENTS:\n"
"When you call any tool with a free-form text argument (search queries, lookup "
"strings, question fields — whatever the tool calls them), the string you pass "
@@ -205,7 +208,7 @@ Any phrasing that requests information about a named entity is a search trigger
Only skip the lookup if you can state concrete facts about the exact entity (title, year, creator, plot) without guessing. A diary or memory mention of the entity's name only confirms the topic came up — it does NOT give you facts you can state. Never invent plot, cast, release year, themes, or other specifics from prior knowledge. If you do not have facts from a tool result in this turn, you must call webSearch.
ARGUMENTS THE TOOL CAN AUTO-DERIVE:
If a tool's description says it has a default for some argument (for example getWeather uses the user's current location when none is given), call the tool in the SAME turn with whatever arguments you do have — even zero — and let the tool fill the rest. Do NOT ask the user to supply that argument. Do NOT reply with a clarifying question like "which location?" or "where are you?" when the tool's description already states it auto-derives that argument. Concretely: a message like "how's the weather today" must trigger getWeather immediately with no arguments, NOT a question back to the user. Asking for an argument the tool auto-derives wastes a turn and frustrates the user.
If a tool's description says it has a default for some argument (for example getWeather uses the user's current location when none is given), call the tool in the SAME turn with whatever arguments you do have — even zero — and let the tool fill the rest. What the user has told you counts as having it: if the conversation or what you remember about the user gives the value, pass it (a user who told you they live in Leeds asking "how's the weather" calls getWeather with location Leeds). Do NOT ask the user to supply that argument. Do NOT reply with a clarifying question like "which location?" or "where are you?" when the tool's description already states it auto-derives that argument. Concretely: a message like "how's the weather today" with no place known must trigger getWeather immediately with no arguments, NOT a question back to the user. Asking for an argument the tool auto-derives wastes a turn and frustrates the user.
SELF-CONTAINED TOOL ARGUMENTS:
Whenever you call any tool with a free-form text argument (a search query, lookup string, question field — whatever the tool names it), the string you pass MUST be a self-contained restatement of the user's intent. Resolve pronouns, ellipsis, and implicit references from earlier turns yourself — the tool does NOT see the conversation history, it only sees the argument you pass. If the previous turn was about "Harry Styles" and the user now asks "what are his most famous songs?", the argument must be something like "Harry Styles most famous songs", NOT "what are his most famous songs". Prefer a compact keyword phrase over a conversational sentence. Never pass the user's literal utterance through when it contains unresolved pronouns, "that", "those", "it", "his", "her", "their", or similar references. This applies to every tool — webSearch, Wikipedia, MCP tools, all of them."""

View File

@@ -96,6 +96,7 @@ Design principles enforced by the engine:
- `thinking` field: Internal reasoning (not shown to user), continue loop
- `content` field: Natural language response to user
- Note: System messages are NOT added after the conversation starts, as this breaks native tool calling in models like Llama 3.2
- Repeat guard: an exact repeat of a tool call (same name and arguments) in one reply, or a third call of the same tool within the reply's last 10 messages, gets guidance to use the earlier results instead of running. Only this reply's messages count: tool results carried over from the previous reply never block a retry the user asks for.
Malformed-response guard (all models):
- After each turn, before the content is accepted as a final reply, `_is_malformed_json_response` checks for structured-data hallucinations that should never reach the user:

View File

@@ -203,8 +203,11 @@ class WeatherTool(Tool):
"Weather only (current + forecast). NOT for time-of-day, date, or "
"location questions — those are already in the assistant's context. "
"Use for ANY weather question: now, later today, tomorrow, this week. "
"Call with {} — user location is auto-detected. Do NOT ask the user "
"where they are or request a city; just call this tool with empty args."
"Set location to the place the user asked about. If they named none, "
"set it to where the user has said they are, from the conversation or "
"what you remember about them. Otherwise leave it empty: the tool "
"auto-detects the user's location (do not copy the detected location "
"into it). Do NOT ask the user where they are; just call this tool."
)
@property
@@ -214,7 +217,7 @@ class WeatherTool(Tool):
"properties": {
"location": {
"type": "string",
"description": "OPTIONAL. City name or location (e.g., 'London', 'New York', 'Tokyo'). Only set this if the user explicitly named a place different from their own location. If omitted, the tool auto-uses the user's current detected location — never ask the user for this argument."
"description": "City name or location (e.g., 'London', 'New York', 'Tokyo'): the place the user asked about, or else where the user has said they are (the conversation or your memory of them). Omit otherwise; the tool then auto-detects the user's location. Never ask the user for this argument."
}
},
"required": []
@@ -364,6 +367,20 @@ class WeatherTool(Tool):
if geo_data.get("results"):
location_str = romanised
# A "district, city, country" name (often the detected location
# copied from the context) may not match as a whole. Retry
# without the leading parts, narrowest area first.
parts = [p.strip() for p in location_str.split(",") if p.strip()]
for i in range(1, len(parts)):
if geo_data.get("results"):
break
wider = ", ".join(parts[i:])
debug_log(f" 🌤️ geocode empty; retrying wider area '{wider}'", "tools")
geocode_params["name"] = wider
geo_response = requests.get(geocode_url, params=geocode_params, timeout=10)
geo_response.raise_for_status()
geo_data = geo_response.json()
if not geo_data.get("results"):
return ToolExecutionResult(
success=False,
@@ -442,8 +459,7 @@ class WeatherTool(Tool):
# The reply is the clean Korean sentence ONLY — no English/°C source
# for the model to echo ("25도 Celsius"), no forecast firehose to
# ramble over. The deterministic weather path in the engine returns
# this verbatim; on the LLM path the model just echoes one sentence.
# ramble over. The model just echoes this one sentence.
lines = [ko_sentence]
reply_text = "\n".join(lines)

View File

@@ -524,3 +524,32 @@ def test_direct_exec_fires_despite_prior_query_tool_carryover(
"direct-exec must fire for the current plan's getWeather step even when "
"prior-query tool results are present in dialogue carryover"
)
def test_planner_receives_live_location_context(mock_config, db, dialogue_memory):
"""The planner gets the same live time/location line as the rest of
the turn, so it can tell an unknown location from a known one."""
from jarvis.reply import engine as engine_mod
mock_config.ollama_chat_model = "gpt-oss:20b"
mock_config.evaluator_enabled = False
mock_config.location_enabled = False
seen: dict = {}
def fake_plan(*args, **kwargs):
seen.update(kwargs)
return ["Reply to the user."]
with patch.object(engine_mod, "chat_with_messages",
return_value=_assistant_content("Hello.")), \
patch.object(engine_mod, "select_tools", return_value=["stop"]), \
patch.object(engine_mod, "plan_query", side_effect=fake_plan):
engine_mod.run_reply_engine(
db=db, cfg=mock_config, tts=None,
text="how is the weather today",
dialogue_memory=dialogue_memory,
)
assert seen.get("live_context") == engine_mod._live_time_location_string(mock_config)
assert "Location: Disabled" in seen["live_context"]

View File

@@ -225,3 +225,53 @@ def test_tool_carryover_text_tool_mode(
assert any(
"Paris" in (m.get("content") or "") for m in tool_like
)
@pytest.mark.unit
@patch("src.jarvis.reply.engine.plan_query", return_value=[])
@patch("src.jarvis.reply.engine.extract_search_params_for_memory", return_value={})
@patch("src.jarvis.reply.engine.run_tool_with_retries")
@patch("src.jarvis.reply.engine.extract_text_from_response")
@patch("src.jarvis.reply.engine.chat_with_messages")
def test_retry_asked_in_next_reply_runs_despite_prior_reply_calls(
mock_chat, mock_extract, mock_tool, _mock_extract, _mock_plan
):
"""Calls carried over from the previous reply do not count against the
per-reply repeat guard: when reply 1 tried a tool twice and the user
then asks to try again, reply 2 runs it."""
def _call(cid, place):
return {"message": {"content": "", "tool_calls": [{
"id": cid, "type": "function",
"function": {"name": "getWeather", "arguments": {"location": place}},
}]}}
mock_tool.side_effect = [
Mock(success=False, reply_text="", error_message="Could not find location."),
Mock(success=False, reply_text="", error_message="Could not find location."),
Mock(success=True, reply_text="Kazbegi: 2.5C, partly cloudy.", error_message=None),
]
mock_chat.side_effect = [
_call("c1", "Kazbegi"),
_call("c2", "Stepantsminda"),
{"message": {"content": "I could not find that place."}},
_call("c3", "Kazbegi, Georgia"),
{"message": {"content": "It is 2.5C and partly cloudy in Kazbegi."}},
]
mock_extract.side_effect = [
"", "", "I could not find that place.",
"", "It is 2.5C and partly cloudy in Kazbegi.",
]
db = Mock()
cfg = _mock_cfg()
dm = DialogueMemory()
run_reply_engine(db=db, cfg=cfg, tts=None,
text="how's the weather in Kazbegi today?",
dialogue_memory=dm)
reply = run_reply_engine(db=db, cfg=cfg, tts=None,
text="just try checking the weather one more time",
dialogue_memory=dm)
assert mock_tool.call_count == 3
assert "2.5" in (reply or "")

View File

@@ -194,6 +194,32 @@ class TestPlanQuery:
)
assert steps == ["Reply to user."]
def test_planner_sees_live_context(self):
"""The planner is told what the live context knows (time, location),
so it can plan a memory lookup for a fact about the user, such as
where they are, only when the live context lacks it."""
cfg = _cfg()
with patch.object(
planner_mod, "call_llm_direct", return_value="Reply to user."
) as llm:
plan_query(
cfg,
"how is the weather today",
"",
[("getWeather", "Weather. (args: location)")],
live_context="Current local time: Monday 09:00. Location: Unknown",
)
assert "Location: Unknown" in llm.call_args.kwargs["user_content"]
def test_prompt_plans_memory_for_user_facts_missing_from_live_context(self):
"""The prompt must tie the searchMemory directive to a user fact
(where they are) that the live context does not give, so a
location-dependent tool still gets the user's place from memory
when auto-detection is off."""
prompt = planner_mod._PROMPT_TEMPLATE
assert "LIVE CONTEXT" in prompt
assert "Location: Unknown" in prompt
def test_prompt_warns_against_fabricating_optional_arguments(self):
"""The planner prompt must explicitly tell the model to omit
optional arguments when the user didn't supply a value, and warn

View File

@@ -163,6 +163,40 @@ class TestWeatherTool:
# The romanised name must be what the second geocode actually queried.
assert mock_get.call_args_list[1].kwargs["params"]["name"] == "Seoul"
@patch('requests.get')
def test_run_comma_place_falls_back_to_wider_area(self, mock_get):
"""A "district, city, country" name the geocoder can't match is retried
without its leading parts, so a location taken from the context (e.g.
a detected "Gangnam-gu, Seoul, South Korea") still gets weather."""
known = {
"Seoul, South Korea": {
"latitude": 37.566, "longitude": 126.9784,
"name": "Seoul", "country": "South Korea", "admin1": "Seoul",
},
}
def fake_get(url, params=None, timeout=None):
resp = Mock()
resp.raise_for_status = Mock()
if "geocoding" in url:
place = known.get(params["name"])
resp.json.return_value = {"results": [place]} if place else {}
else:
resp.json.return_value = {
"current": {
"temperature_2m": 12.0, "apparent_temperature": 11.0,
"weather_code": 0,
},
}
return resp
mock_get.side_effect = fake_get
result = self.tool.run({"location": "Gangnam-gu, Seoul, South Korea"}, self.context)
assert result.success is True
assert "Seoul" in result.reply_text
def test_romanise_place_skips_ascii_names(self):
"""ASCII names already geocode, so no LLM round-trip is spent on them."""
cfg = Mock()