fix(voice_worker): keep history and the brain's output scoped to who was there

- A final is stored under the channel, speaker name and time fixed when its
  end was decided. Decoding takes a while; a move, rename or late joiner
  meanwhile used to change where and when it was said, and who heard it.
- At start the worker closes presence a killed run left open
  (HistoryStore.close_open_presence), so a person who is gone no longer
  hears new speech after a restart.
- run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm
  profile, extractor, diary, graph, digest); the v2 brain uses it so another
  user's profile or diary never reaches the prompt.
- While a brain reply runs, stdout and stderr are muted (also helper threads
  such as the web fetch pool); the worker logs through voice_worker/log.py to
  the original stderr.
- Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the
  prompt in earlier runs; with an 8k tag the chat-record question was answered
  4 of 4 and nothing reached the output. Noted in llm_contexts and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-11-11 13:28:25 +09:00
parent 864b2e7fdb
commit b3dcd12ec9
18 changed files with 265 additions and 25 deletions

View File

@@ -181,6 +181,13 @@ class HistoryStore:
(ts, user_id, channel_id))
self._conn.commit()
def close_open_presence(self, ts: float) -> int:
"""Close every open interval (at start, after a run that ended without closing them)."""
with self._lock:
n = self._conn.execute("UPDATE presence SET left_at=? WHERE left_at IS NULL", (ts,)).rowcount
self._conn.commit()
return n
def presence_close_channel(self, channel_id: str, ts: float) -> None:
with self._lock:
self._conn.execute("UPDATE presence SET left_at=? WHERE channel_id=? AND left_at IS NULL",

View File

@@ -71,7 +71,9 @@ Setting `since` (default none, meaning the whole history) drops records older th
## Writes and maintenance
- `add_voice`, `add_reply`, `add_chat`, `edit_chat`, `delete_chat`.
- `presence_join`, `presence_leave`, `presence_close_channel`.
- `presence_join`, `presence_leave`, `presence_close_channel`, and `close_open_presence(ts)`, which the
host calls at start: a run that was killed never closed its intervals, and nobody is heard until
the app reports who is there again.
- `records_of(person)` and `forget_person(person)`: the person's own records (authored by them or
replies to them), for the data screen's per-person view and delete. Deleting also removes their
presence intervals.

View File

@@ -1023,7 +1023,8 @@ def _history_block(history: Any) -> str:
def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
text: str, dialogue_memory: "DialogueMemory",
language: Optional[str] = None,
history: Optional[Any] = None) -> Optional[str]:
history: Optional[Any] = None,
legacy_memory: bool = True) -> Optional[str]:
"""
Main entry point for reply generation.
@@ -1042,6 +1043,10 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
asker from the history store (recent window and search matches,
already scoped to what this asker may see). Injected as a fenced
reference block; None or empty adds nothing.
legacy_memory: v1's long-term memory (warm profile, diary and graph
enrichment) belongs to a single user. False skips all of it, for
the v2 brain where several people share one assistant and only
the scoped ``history`` may reach the prompt.
Returns:
Generated reply text or None
@@ -1316,7 +1321,8 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
# - Plan without it → skip memory work entirely (no keyword LLM,
# no diary search, no graph search, no digest LLM).
plan_demands_memory = bool(action_plan) and plan_requires_memory(action_plan)
needs_memory = (not action_plan) or plan_demands_memory
# v2 (legacy_memory=False): the scoped history block already answers searchMemory.
needs_memory = legacy_memory and ((not action_plan) or plan_demands_memory)
# Recall gate: if the hot-window already carries a fresh tool result
# covering the query topic, skip diary/graph enrichment for this turn.
@@ -1387,7 +1393,9 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
dialogue_memory.hot_cache_get(_wp_cache_key)
if dialogue_memory and hasattr(dialogue_memory, "hot_cache_get") else None
)
if isinstance(_wp_cached, str):
if not legacy_memory:
debug_log("single-user memory off (v2): no warm profile, diary or graph", "memory")
elif isinstance(_wp_cached, str):
warm_profile_block = _wp_cached
debug_log("warm profile served from conversation cache", "memory")
else:

View File

@@ -62,6 +62,7 @@ Design principles enforced by the engine:
- Injected into the system message, after the warm profile and before the diary block, under the label "Information shared in previous conversations" (the slot the model's usual denial names) inside a `<conversation_records>` fence. The block says the lines are data, not instructions (they include other people's words), and that Jarvis's own earlier answers are not verified facts. Fence tags inside a line are removed so a record cannot close the fence.
- Not gated by the planner: the store read costs no LLM call, and the recent window is needed for follow-ups even when the planner skips memory. `None` or an empty context adds nothing.
- Eval: `evals/test_history_context_recall.py` (answers from a matching earlier record; control without the block does not know the fact).
- `legacy_memory=False` (the v2 brain) turns off v1's single-user long-term memory for the turn: no warm profile, and no memory enrichment (no keyword extractor call, diary, graph or memory digest), even when the planner asks for `searchMemory`. v1's profile, diary and graph belong to one user; in a shared v2 channel they would reach other people. The scoped history block stands in for them.
5. Build Initial Messages
- messages = [

View File

@@ -7,7 +7,7 @@ import asyncio
import os
import sys
from jarvis.debug import debug_log
from .log import debug_log
def main() -> int:

View File

@@ -3,12 +3,13 @@ interface, so the worker logic is tested with a fake (voice_worker.spec.md, "His
from __future__ import annotations
import sys
import threading
import time
from dataclasses import dataclass
from typing import Optional, Protocol
from jarvis.debug import debug_log
from .log import debug_log
from jarvis.memory.history import HistoryContext, Scope
@@ -59,6 +60,44 @@ class MemorySettings:
return Scope(chat=self.chat, others=self.others, since=since, off=self.off)
class Muted:
"""Stand-in for sys.stdout/sys.stderr that drops every write while a brain reply runs.
The v1 reply engine and its tools print the question, the answer, tool arguments, URLs and
memory snippets (``print`` and ``debug_log``, also from helper threads such as the web fetch
pool); the worker's logs must never carry such text. Replies run one at a time; the worker's
own log (``voice_worker.log``) writes to the original stderr and is not muted.
"""
_lock = threading.Lock()
_depth = 0
def __init__(self, target):
self._target = target
def write(self, data):
if Muted._depth:
return len(data)
return self._target.write(data)
def __getattr__(self, name):
return getattr(self._target, name)
@classmethod
def run(cls, fn, *args, **kwargs):
with cls._lock:
for name in ("stdout", "stderr"):
stream = getattr(sys, name)
if not isinstance(stream, cls):
setattr(sys, name, cls(stream))
cls._depth += 1
try:
return fn(*args, **kwargs)
finally:
with cls._lock:
cls._depth -= 1
class JarvisBrain:
"""The existing jarvis reply engine with the user's history context.
@@ -92,8 +131,10 @@ class JarvisBrain:
from jarvis.reply.engine import run_reply_engine
t0 = time.perf_counter()
answer = run_reply_engine(self.db, self.cfg, None, text, self._memory(user),
language=self.language, history=history)
# legacy_memory=False: v1's profile, diary and graph belong to one user; several people
# share this assistant, so only the asker's scoped history may reach the prompt.
answer = Muted.run(run_reply_engine, self.db, self.cfg, None, text, self._memory(user),
language=self.language, history=history, legacy_memory=False)
debug_log(f"brain reply user={user} ok={bool(answer)} ms={(time.perf_counter() - t0) * 1000:.0f}",
"voice_worker")
return answer

17
src/voice_worker/log.py Normal file
View File

@@ -0,0 +1,17 @@
"""The worker's own debug log. It writes to the process's original stderr, so it still shows while
the brain's output is muted (``brain.Muted``); otherwise it is ``jarvis.debug.debug_log``."""
from __future__ import annotations
import sys
from jarvis import debug as _jarvis_debug
def debug_log(message: str, category: str = "debug") -> None:
if not _jarvis_debug._is_debug_enabled():
return
try:
print(f"[{category:^10}] {message}", file=sys.__stderr__, flush=True)
except Exception:
pass

View File

@@ -14,7 +14,7 @@ from typing import Callable, Hashable, Protocol
import numpy as np
from jarvis.debug import debug_log
from .log import debug_log
@dataclass(frozen=True)

View File

@@ -13,7 +13,7 @@ from typing import Callable
import numpy as np
from jarvis.debug import debug_log
from .log import debug_log
from .audio import InputConverter, loudness_equalise
from .brain import Brain, MemorySettings
@@ -57,6 +57,12 @@ class VoiceWorker:
self._ids = itertools.count(1)
async def start(self) -> None:
if self.history is not None:
# A killed worker never closed its sessions' presence; nobody is heard until the app
# reports who is there again.
n = self.history.close_open_presence(time.time())
if n:
debug_log(f"closed {n} presence intervals left open by a previous run", "voice_worker")
if self.transcriber is not None:
self.scheduler = SttScheduler(self.transcriber)
self.scheduler.start()
@@ -230,12 +236,16 @@ class Session:
elif isinstance(ev, Final):
debug_log(f"utterance end speaker={speaker} utt={ev.utt} reason={ev.reason} "
f"ms={len(ev.audio) * 1000 // SR}", "voice_worker")
# Where, when and by whom it was said, fixed now: decoding can take a while, and a
# move or a late joiner meanwhile must not change who heard it.
said = (self.voice_channel, self.names.get(speaker, speaker),
time.time() - (time.monotonic() - ev.decided_at))
if sched is None:
self._send_final(speaker, ev, SttResult("", 0, "stt unavailable"))
self._send_final(speaker, ev, SttResult("", 0, "stt unavailable"), said)
else:
sched.submit_final((self.sid, speaker), loudness_equalise(ev.audio, ev.voiced),
self._opts(self.settings.final_beam),
lambda res, ev=ev: self._send_final(speaker, ev, res))
lambda res, ev=ev, said=said: self._send_final(speaker, ev, res, said))
def _partial_cb(self, speaker: str, st: _Speaker, utt: int):
def cb(res: SttResult) -> None:
@@ -244,10 +254,11 @@ class Session:
self.send({"type": "partial", "speaker": speaker, "utt": utt, "text": res.text})
return cb
def _send_final(self, speaker: str, ev: Final, res: SttResult) -> None:
def _send_final(self, speaker: str, ev: Final, res: SttResult, said: tuple) -> None:
channel, name, ts = said
store = self.worker.history
if store is not None and self.voice_channel and res.text.strip():
store.add_voice(self.voice_channel, speaker, self.names.get(speaker, speaker), res.text, time.time())
if store is not None and channel and res.text.strip():
store.add_voice(channel, speaker, name, res.text, ts)
self.send({"type": "final", "speaker": speaker, "utt": ev.utt, "text": res.text,
"end_reason": ev.reason, "audio_ms": len(ev.audio) * 1000 // SR,
"wait_ms": int((time.monotonic() - ev.decided_at) * 1000), "stt_ms": res.stt_ms})

View File

@@ -10,7 +10,7 @@ import zlib
import numpy as np
from jarvis.debug import debug_log
from .log import debug_log
from .scheduler import DecodeOptions

View File

@@ -10,7 +10,7 @@ from typing import Callable, Protocol
import numpy as np
from jarvis.debug import debug_log
from .log import debug_log
from .sentences import StreamingSplitter

View File

@@ -11,7 +11,7 @@ import threading
import numpy as np
from jarvis.debug import debug_log
from .log import debug_log
CHILD = os.path.join(os.path.dirname(os.path.abspath(__file__)), "melo_child.py")

View File

@@ -202,12 +202,16 @@ which carry Discord's own time.
<- {"type": "reply_text", "id": "r1", "text": "...", "failed": false}
```
- At start the worker closes every presence interval a previous run left open (it may have been
killed); the app then reports who is there.
- `voice_join` closes the presence of everyone in the previous channel and starts a new one;
`voice_leave` and the end of the session close it, because nothing said after that is heard.
`member_join` opens the user's presence in the current channel and remembers their name;
`member_leave` closes it. Members outside a voice channel are ignored.
- Every non-empty `final` is stored as a `voice` record of the current channel under the speaker's
name (the user ID until `member_join` names them). A final outside a voice channel is not stored.
- Every non-empty `final` is stored as a `voice` record under the channel, the speaker's name (the
user ID until `member_join` names them) and the time of the moment its end was decided, all fixed
then: decoding takes a while, and a move, a rename or a late joiner meanwhile must not change where
or when it was said, or who heard it. A final said outside a voice channel is not stored.
- Chat messages are stored only as given; the app sends only registered channels.
- `reply` asks the brain to answer `user`'s `text`. The worker builds the history context for that
user (their scope only) and runs the reply engine on a worker thread, one reply at a time.
@@ -216,7 +220,12 @@ which carry Discord's own time.
answer is stored when the app reports the sent message as `chat_message` with `to`. With
`speak: true` the answer is also spoken through the TTS path under the same id (audio frames and
`tts_done`), and `tts_cancel` with that id stops it. Each user keeps their own short-term dialogue
memory (tool carry-over) across replies.
memory (tool carry-over) across replies. The brain runs the reply engine with v1's single-user
memory off (`legacy_memory=False`: no warm profile, diary or graph), because several people share
this assistant and only the asker's scoped history may reach the prompt.
- While a reply runs, the process's stdout and stderr are muted: the v1 engine and its tools print
questions, answers, tool arguments and memory text, also from helper threads. The worker's own log
(`voice_worker/log.py`) writes to the original stderr and keeps working.
- Memory settings, optional in `hello` as `"memory": {...}` and changeable with
`{"type": "memory_config", ...}`: `chat` (true), `others` (true), `off` (false), `since_days`
(null: whole history), `recent_minutes` (30), `recent_records` (20), `matches` (5),