fix(reply): done-by-user guard keeps tool calls, other people's lines and started tools safe
- A native tool call message with text is not an earlier answer: leaving it out orphaned its role=tool result. Only plain assistant answers are left out. - The guard is off once any tool run started in the reply, whatever it returned: an MCP call with an empty result was not in invoked_tools_history, so a second try could run it again. - HistoryContext.answers marks the lines that are Jarvis's answers to the asker (to_id is the asker); only those leave the history, not someone else saying the same words. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -198,10 +198,10 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
|
||||
## 16. Done-by-user judgements (v2 reply engine)
|
||||
|
||||
- **File**: [src/jarvis/reply/done_guard.py](src/jarvis/reply/done_guard.py) — `superseded()`; called by `run_reply_engine` after the answer is made, before it is kept ([reply.spec.md](src/jarvis/reply/reply.spec.md), "Earlier answers after the user did it (v2)").
|
||||
- **Trigger**: v2 replies (a history context) with no tool call in the reply and an earlier Jarvis answer in the asker's short-term dialogue memory (follow-up turns). The second question runs only after a yes to the first. Not on the second try.
|
||||
- **Trigger**: v2 replies (a history context) in which no tool run was started, with an earlier plain Jarvis answer in the asker's short-term dialogue memory (follow-up turns). The second question runs only after a yes to the first. Not on the second try.
|
||||
- **Model**: `ollama_chat_model` (Strata in v2), non-thinking, temperature 0, timeout 5 s.
|
||||
- **Inputs**: 1) the user's utterance fenced in `<utterance>`: do they say they themselves already did, changed, set or fixed something; 2) the answer fenced in `<reply>`: does it say something is not supported, not possible, or that the assistant cannot do it. Data only, never instructions.
|
||||
- **Output**: one word yes/no each (anything else is no). Two yeses: the reply is made once more ([1] again, plus [7] from the hot-window cache) with Jarvis's earlier answers left out of its context (`left_out`).
|
||||
- **Output**: one word yes/no each (anything else is no). Two yeses: the reply is made once more ([1] again, plus [7] from the hot-window cache) with Jarvis's earlier plain answers and its answer lines in the history context (`HistoryContext.answers`) left out of its context (`left_out`).
|
||||
- **Eval**: `evals/test_said_not_asked.py`.
|
||||
|
||||
---
|
||||
@@ -226,7 +226,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
|
||||
| 13 | Plan step resolver | 0-N (SMALL only) | auto by size + plan | SMALL (via router chain) |
|
||||
| 14 | Tool-specific | per-tool | n/a | LARGE |
|
||||
| 15 | Voice call judgements | 0 (per utterance / tie, outside the reply) | call mode / policy | LARGE (chat model) |
|
||||
| 16 | Done-by-user judgements | 0-2 (v2 follow-up turns without a tool call; two yeses add a second reply) | No | LARGE (chat model) |
|
||||
| 16 | Done-by-user judgements | 0-2 (v2 follow-up turns without a tool run; two yeses add a second reply) | No | LARGE (chat model) |
|
||||
|
||||
## Size-aware auto switches
|
||||
|
||||
|
||||
@@ -33,6 +33,7 @@ FIELD_HISTORY = [
|
||||
f"[2026-11-18 18:27] 사금향: {FIELD_QUESTION}",
|
||||
f"[2026-11-18 18:27] 자비스: {FIELD_REFUSAL}",
|
||||
]
|
||||
FIELD_ANSWERS = [FIELD_HISTORY[1]] # the history line that is Jarvis's answer to the asker
|
||||
# The same turn in the asker's short-term dialogue memory, as the v2 brain keeps it.
|
||||
FIELD_MEMORY = [("user", FIELD_QUESTION), ("assistant", FIELD_REFUSAL)]
|
||||
|
||||
@@ -97,7 +98,8 @@ def _run(mock_config, eval_db, eval_dialogue_memory, query, recent, memory=()):
|
||||
turn_state.set_chat(chat)
|
||||
try:
|
||||
reply = run_reply_engine(db=eval_db, cfg=mock_config, tts=None, text=query,
|
||||
dialogue_memory=eval_dialogue_memory, history=HistoryContext(recent=recent),
|
||||
dialogue_memory=eval_dialogue_memory,
|
||||
history=HistoryContext(recent=recent, answers=[x for x in FIELD_ANSWERS if x in recent]),
|
||||
legacy_memory=False, default_language="Korean", compact_prompt=True,
|
||||
intent_judge=True) or ""
|
||||
finally:
|
||||
|
||||
@@ -101,10 +101,12 @@ class Trace:
|
||||
|
||||
@dataclass
|
||||
class HistoryContext:
|
||||
"""Lines for the prompt: the recent window and the search matches, kept apart for labelling."""
|
||||
"""Lines for the prompt: the recent window and the search matches, kept apart for labelling.
|
||||
``answers``: which of those lines are Jarvis's answers to the asker (not shown on its own)."""
|
||||
|
||||
recent: List[str] = field(default_factory=list)
|
||||
matches: List[str] = field(default_factory=list)
|
||||
answers: List[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _steps_json(steps: list) -> str:
|
||||
@@ -425,4 +427,5 @@ class HistoryStore:
|
||||
debug_log(f"history context: {len(recent)} recent, {len(matches)} matches, "
|
||||
f"~{total:.0f} tokens", "memory")
|
||||
return HistoryContext(recent=[self.format_line(r) for r in recent],
|
||||
matches=[self.format_line(r) for r in matches])
|
||||
matches=[self.format_line(r) for r in matches],
|
||||
answers=[self.format_line(r) for r in recent + matches if r.to_id == asker])
|
||||
|
||||
@@ -66,10 +66,11 @@ Setting `since` (default none, meaning the whole history) drops records older th
|
||||
matches may use at most half the budget (lowest score cut first), so a long recent window cannot
|
||||
crowd out the records the question is about; the recent window gets the rest (oldest cut first).
|
||||
A record that is a match is not repeated in the recent block. Returns the two blocks separately
|
||||
so the prompt can label them. The reply engine may leave Jarvis's earlier answers out of one
|
||||
answer's copy of the blocks when the user says they did something themselves
|
||||
(`src/jarvis/reply/reply.spec.md`, "Earlier answers after the user did it (v2)"); the store is
|
||||
not changed.
|
||||
so the prompt can label them, and `answers`: which of their lines are Jarvis's answers to the
|
||||
asker (`reply` or `chat` records whose `to_id` is the asker), never shown as a block of its own.
|
||||
The reply engine may leave those lines out of one answer's copy of the blocks when the user says
|
||||
they did something themselves (`src/jarvis/reply/reply.spec.md`, "Earlier answers after the user
|
||||
did it (v2)"); the store is not changed.
|
||||
|
||||
## Traces
|
||||
|
||||
|
||||
@@ -45,23 +45,21 @@ def _yes_no(cfg, system: str, user_content: str, what: str, timeout_sec: float)
|
||||
return result
|
||||
|
||||
|
||||
def _line_text(line: str) -> str:
|
||||
"""The text of a history line ``[date time] name: text``."""
|
||||
head, sep, rest = line.partition("] ")
|
||||
if not sep or not head.startswith("["):
|
||||
return line
|
||||
return rest.partition(": ")[2] or rest
|
||||
def _is_answer(m: dict) -> bool:
|
||||
"""A plain answer of Jarvis: not a native tool call message (its tool results answer it)."""
|
||||
return (m.get("role") == "assistant" and not m.get("tool_calls") and isinstance(m.get("content"), str)
|
||||
and bool(m["content"].strip()))
|
||||
|
||||
|
||||
def earlier_answers(recent_messages: Iterable[dict], history: Any) -> list[str]:
|
||||
"""Jarvis's earlier answers in the short-term dialogue memory, and the history lines (recent and
|
||||
matches) with the same text, as given, so a second try can leave exactly these out."""
|
||||
answers = [m["content"] for m in recent_messages or []
|
||||
if m.get("role") == "assistant" and isinstance(m.get("content"), str) and m["content"].strip()]
|
||||
said = {a.strip() for a in answers}
|
||||
lines = [x for x in [*(getattr(history, "recent", None) or []), *(getattr(history, "matches", None) or [])]
|
||||
if isinstance(x, str) and _line_text(x).strip() in said]
|
||||
return answers + lines if answers else []
|
||||
"""Jarvis's earlier plain answers in the short-term dialogue memory, and its answer lines in the
|
||||
history context (``HistoryContext.answers``, the lines that are its answers to the asker), as
|
||||
given, so a second try can leave exactly these out. Empty without an answer in the memory."""
|
||||
answers = [m["content"] for m in recent_messages or [] if _is_answer(m)]
|
||||
if not answers:
|
||||
return []
|
||||
shown = {*(getattr(history, "recent", None) or []), *(getattr(history, "matches", None) or [])}
|
||||
return answers + [x for x in getattr(history, "answers", None) or [] if x in shown]
|
||||
|
||||
|
||||
def superseded(cfg, text: str, reply: str, timeout_sec: float = 5.0) -> bool:
|
||||
|
||||
@@ -1251,15 +1251,15 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
|
||||
# Done-by-user guard's second try (v2): Jarvis's earlier answers are left out.
|
||||
if left_out:
|
||||
from .done_guard import _is_answer
|
||||
_gone = set(left_out)
|
||||
recent_messages = [m for m in recent_messages
|
||||
if not (m.get("role") == "assistant" and isinstance(m.get("content"), str)
|
||||
and m["content"] in _gone)]
|
||||
recent_messages = [m for m in recent_messages if not (_is_answer(m) and m["content"] in _gone)]
|
||||
if history is not None:
|
||||
from ..memory.history import HistoryContext
|
||||
history = HistoryContext(
|
||||
recent=[x for x in (getattr(history, "recent", None) or []) if x not in _gone],
|
||||
matches=[x for x in (getattr(history, "matches", None) or []) if x not in _gone])
|
||||
matches=[x for x in (getattr(history, "matches", None) or []) if x not in _gone],
|
||||
answers=[x for x in (getattr(history, "answers", None) or []) if x not in _gone])
|
||||
|
||||
# New conversation reset: when the previous session lapsed past the
|
||||
# inactivity window, drop the conversation-scoped cache and any
|
||||
@@ -2102,6 +2102,8 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
recent_tool_signatures = [] # keep last few tool calls: [(name, stable_args_json)]
|
||||
# Tools actually invoked during this reply — (name, args_summary, result_summary).
|
||||
invoked_tools_history: list[tuple[str, str, str]] = []
|
||||
# Every tool run this reply started, whatever it returned (the done-by-user guard).
|
||||
tools_started = 0
|
||||
# System message with guidance, tools, and enrichment
|
||||
messages.append({"role": "system", "content": _build_initial_system_message()})
|
||||
# Include recent dialogue memory as-is
|
||||
@@ -2444,6 +2446,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
}
|
||||
],
|
||||
})
|
||||
tools_started += 1
|
||||
_plan_result = run_tool_with_retries(
|
||||
db=db,
|
||||
cfg=cfg,
|
||||
@@ -2791,6 +2794,7 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
continue
|
||||
|
||||
# Execute tool
|
||||
tools_started += 1
|
||||
result = run_tool_with_retries(
|
||||
db=db,
|
||||
cfg=cfg,
|
||||
@@ -3158,9 +3162,9 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
|
||||
# Done-by-user guard (v2, reply.spec.md "Earlier answers after the user did it (v2)"): when the
|
||||
# user says they did something themselves and the answer still says something cannot be done,
|
||||
# the answer is made once more without Jarvis's earlier answers. Not after a tool ran: a second
|
||||
# try would run it again.
|
||||
if history is not None and left_out is None and not invoked_tools_history:
|
||||
# the answer is made once more without Jarvis's earlier answers. Not after a tool run started,
|
||||
# whatever it returned: a second try would run it again.
|
||||
if history is not None and left_out is None and not tools_started:
|
||||
from .done_guard import earlier_answers, superseded
|
||||
_earlier = earlier_answers(recent_messages, history)
|
||||
if _earlier and superseded(cfg, redacted, reply):
|
||||
|
||||
@@ -110,8 +110,8 @@ Design principles enforced by the engine:
|
||||
|
||||
Earlier answers after the user did it (v2):
|
||||
- Field turn 2026-11-18: Jarvis said speed control is not supported, the user changed the speed in the app and said "I adjusted your speed, so say something", and Jarvis refused again (in the same or other words). Its own earlier answer in the context outweighed what the user had just said; prompt wording did not move it.
|
||||
- After the answer is made and before it is kept or spoken: when the reply has history (v2), no tool ran in this reply, and the asker's short-term dialogue memory holds an earlier answer of Jarvis, `done_guard.superseded` asks the chat model two short yes/no questions (non-thinking, temperature 0, 5 s, texts fenced as data): does the user say they themselves already did, changed, set or fixed something; only on a yes, does the answer say something is not supported, not possible or that Jarvis cannot do it. On two yeses the answer is dropped and made once more with `left_out`: Jarvis's earlier answers in the dialogue memory and the history lines (recent and matches) with the same text are not in that reply's context. The user's own earlier words stay. The second try keeps whatever it answers and runs no guard; the turn is stored once, with that answer.
|
||||
- Anything else keeps the first answer: a "no" (the user asked a question, asked to repeat, or only asked for something), a non-refusing answer, no usable judgement, no earlier answer, or a tool call in the reply (a second try would run it again).
|
||||
- After the answer is made and before it is kept or spoken: when the reply has history (v2), no tool run was started in this reply (whatever it returned), and the asker's short-term dialogue memory holds an earlier plain answer of Jarvis (not a native tool call message), `done_guard.superseded` asks the chat model two short yes/no questions (non-thinking, temperature 0, 5 s, texts fenced as data): does the user say they themselves already did, changed, set or fixed something; only on a yes, does the answer say something is not supported, not possible or that Jarvis cannot do it. On two yeses the answer is dropped and made once more with `left_out`: Jarvis's earlier plain answers in the dialogue memory and the history lines that are its answers to the asker (`HistoryContext.answers`, history.spec.md) are not in that reply's context. The user's own earlier words, other people's lines (even with the same words) and earlier tool calls with their results stay. The second try keeps whatever it answers and runs no guard; the turn is stored once, with that answer.
|
||||
- Anything else keeps the first answer: a "no" (the user asked a question, asked to repeat, or only asked for something), a non-refusing answer, no usable judgement, no earlier answer, or a tool run started in the reply, even one that returned nothing or failed (a second try would run it again).
|
||||
- Cost: one short call on v2 follow-up turns (an earlier answer in the dialogue memory), a second only after a yes, and a second reply only when both are yes. Logs carry the outcomes, never the text.
|
||||
- Eval: `evals/test_said_not_asked.py` (`speed-adjusted-field` with the field turn's history and dialogue memory; control: "what did you just say? say it again" still says it again).
|
||||
|
||||
|
||||
@@ -28,9 +28,11 @@ def _content(text: str) -> dict:
|
||||
return {"message": {"role": "assistant", "content": text}}
|
||||
|
||||
|
||||
def _run(mock_config, db, dialogue_memory, *, text, chat_replies, done="yes", earlier=True, tool_first=False):
|
||||
def _run(mock_config, db, dialogue_memory, *, text, chat_replies, done="yes", earlier=True, tool_first=False,
|
||||
tool_text="ok", others=()):
|
||||
"""``done``: the judge's word for "the user says they did it"; a reply is judged a refusal when
|
||||
it is one of REFUSALS."""
|
||||
it is one of REFUSALS. ``tool_text``: what the tool returns; ``others``: history lines of other
|
||||
people, after Jarvis's answer."""
|
||||
from jarvis.memory.history import HistoryContext
|
||||
from jarvis.reply import engine as engine_mod
|
||||
from jarvis.tools.types import ToolExecutionResult
|
||||
@@ -41,8 +43,9 @@ def _run(mock_config, db, dialogue_memory, *, text, chat_replies, done="yes", ea
|
||||
if earlier:
|
||||
dialogue_memory.add_message("user", QUESTION)
|
||||
dialogue_memory.add_message("assistant", REFUSAL)
|
||||
history = HistoryContext(recent=[f"[2026-11-18 18:27] 사금향: {QUESTION}",
|
||||
f"[2026-11-18 18:27] 자비스: {REFUSAL}"] if earlier else [])
|
||||
answer_line = f"[2026-11-18 18:27] 자비스: {REFUSAL}"
|
||||
history = HistoryContext(recent=[f"[2026-11-18 18:27] 사금향: {QUESTION}", answer_line, *others]
|
||||
if earlier else [], answers=[answer_line] if earlier else [])
|
||||
seen: list[list[dict]] = []
|
||||
replies = iter(chat_replies)
|
||||
judged: list[str] = []
|
||||
@@ -58,7 +61,7 @@ def _run(mock_config, db, dialogue_memory, *, text, chat_replies, done="yes", ea
|
||||
return "yes" if user_content.removeprefix("<reply>").removesuffix("</reply>") in REFUSALS else "no"
|
||||
|
||||
def fake_tool(db, cfg, tool_name, tool_args, **kwargs):
|
||||
return ToolExecutionResult(success=True, reply_text="ok", error_message=None)
|
||||
return ToolExecutionResult(success=True, reply_text=tool_text, error_message=None)
|
||||
|
||||
routed = ["getWeather", "stop"] if tool_first else ["stop"]
|
||||
with patch.object(engine_mod, "run_tool_with_retries", side_effect=fake_tool), \
|
||||
@@ -126,11 +129,44 @@ def test_no_usable_judgement_keeps_the_answer(mock_config, db, dialogue_memory,
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_a_turn_that_ran_a_tool_is_not_made_again(mock_config, db, dialogue_memory):
|
||||
"""A second try would run the tool (and anything it does) twice."""
|
||||
@pytest.mark.parametrize("tool_text", ["ok", "", None], ids=["result", "empty", "none"])
|
||||
def test_a_turn_that_ran_a_tool_is_not_made_again(mock_config, db, dialogue_memory, tool_text):
|
||||
"""A second try would run the tool (and anything it does) twice, whatever it returned."""
|
||||
call = {"message": {"role": "assistant", "content": "", "tool_calls": [
|
||||
{"id": "call_1", "type": "function", "function": {"name": "getWeather", "arguments": {}}}]}}
|
||||
reply, seen, judged = _run(mock_config, db, dialogue_memory, text=DID_IT,
|
||||
chat_replies=[call, _content(REFUSAL)], tool_first=True)
|
||||
chat_replies=[call, _content(REFUSAL)], tool_first=True, tool_text=tool_text)
|
||||
assert reply == REFUSAL
|
||||
assert judged == [] and len(seen) == 2
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_an_earlier_tool_call_and_its_result_stay_together(mock_config, db, dialogue_memory):
|
||||
"""A native tool call message with text is not an answer to leave out: its tool result would be
|
||||
left without the call it answers."""
|
||||
dialogue_memory.add_message("user", "서울 날씨 어때?")
|
||||
dialogue_memory.record_tool_turn([
|
||||
{"role": "assistant", "content": REFUSAL, "tool_calls": [
|
||||
{"id": "call_0", "type": "function", "function": {"name": "getWeather", "arguments": {}}}]},
|
||||
{"role": "tool", "tool_call_id": "call_0", "content": "Seoul: 13C, clear sky."}])
|
||||
reply, seen, _ = _run(mock_config, db, dialogue_memory, text=DID_IT,
|
||||
chat_replies=[_content(REFUSAL), _content(SPOKEN)])
|
||||
assert reply == SPOKEN and len(seen) == 2
|
||||
second = seen[1]
|
||||
calls = {tc["id"] for m in second if m.get("role") == "assistant" for tc in (m.get("tool_calls") or [])}
|
||||
results = [m["tool_call_id"] for m in second if m.get("role") == "tool"]
|
||||
assert results == ["call_0"] and "call_0" in calls
|
||||
# Jarvis's plain earlier answer is still left out.
|
||||
assert not [m for m in second if m.get("role") == "assistant" and not m.get("tool_calls")
|
||||
and m.get("content") == REFUSAL]
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_only_jarvis_lines_leave_the_history(mock_config, db, dialogue_memory):
|
||||
"""Someone else saying the same words is their record, not Jarvis's answer."""
|
||||
quoted = f"[2026-11-18 18:28] 민수: {REFUSAL}"
|
||||
reply, seen, _ = _run(mock_config, db, dialogue_memory, text=DID_IT,
|
||||
chat_replies=[_content(REFUSAL), _content(SPOKEN)], others=[quoted])
|
||||
assert reply == SPOKEN
|
||||
assert quoted in _said(seen[1])
|
||||
assert "자비스: " + REFUSAL not in _said(seen[1])
|
||||
|
||||
@@ -150,6 +150,20 @@ def test_context_formats_lines_and_keeps_under_budget(store):
|
||||
assert ctx.recent and ctx.recent[-1].endswith("잡담 39 " + "가" * 60)
|
||||
|
||||
|
||||
|
||||
def test_context_marks_jarvis_answers_to_the_asker(store):
|
||||
"""Jarvis's answers to the asker are told apart from the same words said by a person."""
|
||||
store.presence_join("a", VC, T0 - 10)
|
||||
store.add_voice(VC, "a", "Alice", "속도 바꿀 수 있어?", T0)
|
||||
store.add_reply(VC, "a", "지원하지 않습니다", T0 + 1)
|
||||
store.add_voice(VC, "b", "Bob", "지원하지 않습니다", T0 + 2)
|
||||
store.add_reply(VC, "b", "피자 좋네요", T0 + 3)
|
||||
ctx = store.context("a", "속도", now=T0 + 10, scope=Scope())
|
||||
lines = ctx.recent + ctx.matches
|
||||
assert len(lines) == 4
|
||||
assert len(ctx.answers) == 1 and ctx.answers[0] in lines
|
||||
assert ctx.answers[0].endswith(": 지원하지 않습니다") and "Bob" not in ctx.answers[0]
|
||||
|
||||
def test_context_drops_matches_before_recent_lines(store):
|
||||
store.add_voice(VC, "a", "Alice", "old match about cats " + "x" * 300, T0)
|
||||
store.add_voice(VC, "a", "Alice", "recent line", T0 + 5000)
|
||||
|
||||
Reference in New Issue
Block a user