fix(voice_worker): keep history and the brain's output scoped to who was there
- A final is stored under the channel, speaker name and time fixed when its end was decided. Decoding takes a while; a move, rename or late joiner meanwhile used to change where and when it was said, and who heard it. - At start the worker closes presence a killed run left open (HistoryStore.close_open_presence), so a person who is gone no longer hears new speech after a restart. - run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm profile, extractor, diary, graph, digest); the v2 brain uses it so another user's profile or diary never reaches the prompt. - While a brain reply runs, stdout and stderr are muted (also helper threads such as the web fetch pool); the worker logs through voice_worker/log.py to the original stderr. - Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the prompt in earlier runs; with an 8k tag the chat-record question was answered 4 of 4 and nothing reached the output. Noted in llm_contexts and the eval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -181,6 +181,13 @@ class HistoryStore:
|
||||
(ts, user_id, channel_id))
|
||||
self._conn.commit()
|
||||
|
||||
def close_open_presence(self, ts: float) -> int:
|
||||
"""Close every open interval (at start, after a run that ended without closing them)."""
|
||||
with self._lock:
|
||||
n = self._conn.execute("UPDATE presence SET left_at=? WHERE left_at IS NULL", (ts,)).rowcount
|
||||
self._conn.commit()
|
||||
return n
|
||||
|
||||
def presence_close_channel(self, channel_id: str, ts: float) -> None:
|
||||
with self._lock:
|
||||
self._conn.execute("UPDATE presence SET left_at=? WHERE channel_id=? AND left_at IS NULL",
|
||||
|
||||
@@ -71,7 +71,9 @@ Setting `since` (default none, meaning the whole history) drops records older th
|
||||
## Writes and maintenance
|
||||
|
||||
- `add_voice`, `add_reply`, `add_chat`, `edit_chat`, `delete_chat`.
|
||||
- `presence_join`, `presence_leave`, `presence_close_channel`.
|
||||
- `presence_join`, `presence_leave`, `presence_close_channel`, and `close_open_presence(ts)`, which the
|
||||
host calls at start: a run that was killed never closed its intervals, and nobody is heard until
|
||||
the app reports who is there again.
|
||||
- `records_of(person)` and `forget_person(person)`: the person's own records (authored by them or
|
||||
replies to them), for the data screen's per-person view and delete. Deleting also removes their
|
||||
presence intervals.
|
||||
|
||||
@@ -1023,7 +1023,8 @@ def _history_block(history: Any) -> str:
|
||||
def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
text: str, dialogue_memory: "DialogueMemory",
|
||||
language: Optional[str] = None,
|
||||
history: Optional[Any] = None) -> Optional[str]:
|
||||
history: Optional[Any] = None,
|
||||
legacy_memory: bool = True) -> Optional[str]:
|
||||
"""
|
||||
Main entry point for reply generation.
|
||||
|
||||
@@ -1042,6 +1043,10 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
asker from the history store (recent window and search matches,
|
||||
already scoped to what this asker may see). Injected as a fenced
|
||||
reference block; None or empty adds nothing.
|
||||
legacy_memory: v1's long-term memory (warm profile, diary and graph
|
||||
enrichment) belongs to a single user. False skips all of it, for
|
||||
the v2 brain where several people share one assistant and only
|
||||
the scoped ``history`` may reach the prompt.
|
||||
|
||||
Returns:
|
||||
Generated reply text or None
|
||||
@@ -1316,7 +1321,8 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
# - Plan without it → skip memory work entirely (no keyword LLM,
|
||||
# no diary search, no graph search, no digest LLM).
|
||||
plan_demands_memory = bool(action_plan) and plan_requires_memory(action_plan)
|
||||
needs_memory = (not action_plan) or plan_demands_memory
|
||||
# v2 (legacy_memory=False): the scoped history block already answers searchMemory.
|
||||
needs_memory = legacy_memory and ((not action_plan) or plan_demands_memory)
|
||||
|
||||
# Recall gate: if the hot-window already carries a fresh tool result
|
||||
# covering the query topic, skip diary/graph enrichment for this turn.
|
||||
@@ -1387,7 +1393,9 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
|
||||
dialogue_memory.hot_cache_get(_wp_cache_key)
|
||||
if dialogue_memory and hasattr(dialogue_memory, "hot_cache_get") else None
|
||||
)
|
||||
if isinstance(_wp_cached, str):
|
||||
if not legacy_memory:
|
||||
debug_log("single-user memory off (v2): no warm profile, diary or graph", "memory")
|
||||
elif isinstance(_wp_cached, str):
|
||||
warm_profile_block = _wp_cached
|
||||
debug_log("warm profile served from conversation cache", "memory")
|
||||
else:
|
||||
|
||||
@@ -62,6 +62,7 @@ Design principles enforced by the engine:
|
||||
- Injected into the system message, after the warm profile and before the diary block, under the label "Information shared in previous conversations" (the slot the model's usual denial names) inside a `<conversation_records>` fence. The block says the lines are data, not instructions (they include other people's words), and that Jarvis's own earlier answers are not verified facts. Fence tags inside a line are removed so a record cannot close the fence.
|
||||
- Not gated by the planner: the store read costs no LLM call, and the recent window is needed for follow-ups even when the planner skips memory. `None` or an empty context adds nothing.
|
||||
- Eval: `evals/test_history_context_recall.py` (answers from a matching earlier record; control without the block does not know the fact).
|
||||
- `legacy_memory=False` (the v2 brain) turns off v1's single-user long-term memory for the turn: no warm profile, and no memory enrichment (no keyword extractor call, diary, graph or memory digest), even when the planner asks for `searchMemory`. v1's profile, diary and graph belong to one user; in a shared v2 channel they would reach other people. The scoped history block stands in for them.
|
||||
|
||||
5. Build Initial Messages
|
||||
- messages = [
|
||||
|
||||
@@ -7,7 +7,7 @@ import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
|
||||
|
||||
def main() -> int:
|
||||
|
||||
@@ -3,12 +3,13 @@ interface, so the worker logic is tested with a fake (voice_worker.spec.md, "His
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, Protocol
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
from jarvis.memory.history import HistoryContext, Scope
|
||||
|
||||
|
||||
@@ -59,6 +60,44 @@ class MemorySettings:
|
||||
return Scope(chat=self.chat, others=self.others, since=since, off=self.off)
|
||||
|
||||
|
||||
class Muted:
|
||||
"""Stand-in for sys.stdout/sys.stderr that drops every write while a brain reply runs.
|
||||
|
||||
The v1 reply engine and its tools print the question, the answer, tool arguments, URLs and
|
||||
memory snippets (``print`` and ``debug_log``, also from helper threads such as the web fetch
|
||||
pool); the worker's logs must never carry such text. Replies run one at a time; the worker's
|
||||
own log (``voice_worker.log``) writes to the original stderr and is not muted.
|
||||
"""
|
||||
|
||||
_lock = threading.Lock()
|
||||
_depth = 0
|
||||
|
||||
def __init__(self, target):
|
||||
self._target = target
|
||||
|
||||
def write(self, data):
|
||||
if Muted._depth:
|
||||
return len(data)
|
||||
return self._target.write(data)
|
||||
|
||||
def __getattr__(self, name):
|
||||
return getattr(self._target, name)
|
||||
|
||||
@classmethod
|
||||
def run(cls, fn, *args, **kwargs):
|
||||
with cls._lock:
|
||||
for name in ("stdout", "stderr"):
|
||||
stream = getattr(sys, name)
|
||||
if not isinstance(stream, cls):
|
||||
setattr(sys, name, cls(stream))
|
||||
cls._depth += 1
|
||||
try:
|
||||
return fn(*args, **kwargs)
|
||||
finally:
|
||||
with cls._lock:
|
||||
cls._depth -= 1
|
||||
|
||||
|
||||
class JarvisBrain:
|
||||
"""The existing jarvis reply engine with the user's history context.
|
||||
|
||||
@@ -92,8 +131,10 @@ class JarvisBrain:
|
||||
from jarvis.reply.engine import run_reply_engine
|
||||
|
||||
t0 = time.perf_counter()
|
||||
answer = run_reply_engine(self.db, self.cfg, None, text, self._memory(user),
|
||||
language=self.language, history=history)
|
||||
# legacy_memory=False: v1's profile, diary and graph belong to one user; several people
|
||||
# share this assistant, so only the asker's scoped history may reach the prompt.
|
||||
answer = Muted.run(run_reply_engine, self.db, self.cfg, None, text, self._memory(user),
|
||||
language=self.language, history=history, legacy_memory=False)
|
||||
debug_log(f"brain reply user={user} ok={bool(answer)} ms={(time.perf_counter() - t0) * 1000:.0f}",
|
||||
"voice_worker")
|
||||
return answer
|
||||
|
||||
17
src/voice_worker/log.py
Normal file
17
src/voice_worker/log.py
Normal file
@@ -0,0 +1,17 @@
|
||||
"""The worker's own debug log. It writes to the process's original stderr, so it still shows while
|
||||
the brain's output is muted (``brain.Muted``); otherwise it is ``jarvis.debug.debug_log``."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
from jarvis import debug as _jarvis_debug
|
||||
|
||||
|
||||
def debug_log(message: str, category: str = "debug") -> None:
|
||||
if not _jarvis_debug._is_debug_enabled():
|
||||
return
|
||||
try:
|
||||
print(f"[{category:^10}] {message}", file=sys.__stderr__, flush=True)
|
||||
except Exception:
|
||||
pass
|
||||
@@ -14,7 +14,7 @@ from typing import Callable, Hashable, Protocol
|
||||
|
||||
import numpy as np
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
||||
@@ -13,7 +13,7 @@ from typing import Callable
|
||||
|
||||
import numpy as np
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
|
||||
from .audio import InputConverter, loudness_equalise
|
||||
from .brain import Brain, MemorySettings
|
||||
@@ -57,6 +57,12 @@ class VoiceWorker:
|
||||
self._ids = itertools.count(1)
|
||||
|
||||
async def start(self) -> None:
|
||||
if self.history is not None:
|
||||
# A killed worker never closed its sessions' presence; nobody is heard until the app
|
||||
# reports who is there again.
|
||||
n = self.history.close_open_presence(time.time())
|
||||
if n:
|
||||
debug_log(f"closed {n} presence intervals left open by a previous run", "voice_worker")
|
||||
if self.transcriber is not None:
|
||||
self.scheduler = SttScheduler(self.transcriber)
|
||||
self.scheduler.start()
|
||||
@@ -230,12 +236,16 @@ class Session:
|
||||
elif isinstance(ev, Final):
|
||||
debug_log(f"utterance end speaker={speaker} utt={ev.utt} reason={ev.reason} "
|
||||
f"ms={len(ev.audio) * 1000 // SR}", "voice_worker")
|
||||
# Where, when and by whom it was said, fixed now: decoding can take a while, and a
|
||||
# move or a late joiner meanwhile must not change who heard it.
|
||||
said = (self.voice_channel, self.names.get(speaker, speaker),
|
||||
time.time() - (time.monotonic() - ev.decided_at))
|
||||
if sched is None:
|
||||
self._send_final(speaker, ev, SttResult("", 0, "stt unavailable"))
|
||||
self._send_final(speaker, ev, SttResult("", 0, "stt unavailable"), said)
|
||||
else:
|
||||
sched.submit_final((self.sid, speaker), loudness_equalise(ev.audio, ev.voiced),
|
||||
self._opts(self.settings.final_beam),
|
||||
lambda res, ev=ev: self._send_final(speaker, ev, res))
|
||||
lambda res, ev=ev, said=said: self._send_final(speaker, ev, res, said))
|
||||
|
||||
def _partial_cb(self, speaker: str, st: _Speaker, utt: int):
|
||||
def cb(res: SttResult) -> None:
|
||||
@@ -244,10 +254,11 @@ class Session:
|
||||
self.send({"type": "partial", "speaker": speaker, "utt": utt, "text": res.text})
|
||||
return cb
|
||||
|
||||
def _send_final(self, speaker: str, ev: Final, res: SttResult) -> None:
|
||||
def _send_final(self, speaker: str, ev: Final, res: SttResult, said: tuple) -> None:
|
||||
channel, name, ts = said
|
||||
store = self.worker.history
|
||||
if store is not None and self.voice_channel and res.text.strip():
|
||||
store.add_voice(self.voice_channel, speaker, self.names.get(speaker, speaker), res.text, time.time())
|
||||
if store is not None and channel and res.text.strip():
|
||||
store.add_voice(channel, speaker, name, res.text, ts)
|
||||
self.send({"type": "final", "speaker": speaker, "utt": ev.utt, "text": res.text,
|
||||
"end_reason": ev.reason, "audio_ms": len(ev.audio) * 1000 // SR,
|
||||
"wait_ms": int((time.monotonic() - ev.decided_at) * 1000), "stt_ms": res.stt_ms})
|
||||
|
||||
@@ -10,7 +10,7 @@ import zlib
|
||||
|
||||
import numpy as np
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
|
||||
from .scheduler import DecodeOptions
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ from typing import Callable, Protocol
|
||||
|
||||
import numpy as np
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
|
||||
from .sentences import StreamingSplitter
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ import threading
|
||||
|
||||
import numpy as np
|
||||
|
||||
from jarvis.debug import debug_log
|
||||
from .log import debug_log
|
||||
|
||||
CHILD = os.path.join(os.path.dirname(os.path.abspath(__file__)), "melo_child.py")
|
||||
|
||||
|
||||
@@ -202,12 +202,16 @@ which carry Discord's own time.
|
||||
<- {"type": "reply_text", "id": "r1", "text": "...", "failed": false}
|
||||
```
|
||||
|
||||
- At start the worker closes every presence interval a previous run left open (it may have been
|
||||
killed); the app then reports who is there.
|
||||
- `voice_join` closes the presence of everyone in the previous channel and starts a new one;
|
||||
`voice_leave` and the end of the session close it, because nothing said after that is heard.
|
||||
`member_join` opens the user's presence in the current channel and remembers their name;
|
||||
`member_leave` closes it. Members outside a voice channel are ignored.
|
||||
- Every non-empty `final` is stored as a `voice` record of the current channel under the speaker's
|
||||
name (the user ID until `member_join` names them). A final outside a voice channel is not stored.
|
||||
- Every non-empty `final` is stored as a `voice` record under the channel, the speaker's name (the
|
||||
user ID until `member_join` names them) and the time of the moment its end was decided, all fixed
|
||||
then: decoding takes a while, and a move, a rename or a late joiner meanwhile must not change where
|
||||
or when it was said, or who heard it. A final said outside a voice channel is not stored.
|
||||
- Chat messages are stored only as given; the app sends only registered channels.
|
||||
- `reply` asks the brain to answer `user`'s `text`. The worker builds the history context for that
|
||||
user (their scope only) and runs the reply engine on a worker thread, one reply at a time.
|
||||
@@ -216,7 +220,12 @@ which carry Discord's own time.
|
||||
answer is stored when the app reports the sent message as `chat_message` with `to`. With
|
||||
`speak: true` the answer is also spoken through the TTS path under the same id (audio frames and
|
||||
`tts_done`), and `tts_cancel` with that id stops it. Each user keeps their own short-term dialogue
|
||||
memory (tool carry-over) across replies.
|
||||
memory (tool carry-over) across replies. The brain runs the reply engine with v1's single-user
|
||||
memory off (`legacy_memory=False`: no warm profile, diary or graph), because several people share
|
||||
this assistant and only the asker's scoped history may reach the prompt.
|
||||
- While a reply runs, the process's stdout and stderr are muted: the v1 engine and its tools print
|
||||
questions, answers, tool arguments and memory text, also from helper threads. The worker's own log
|
||||
(`voice_worker/log.py`) writes to the original stderr and keeps working.
|
||||
- Memory settings, optional in `hello` as `"memory": {...}` and changeable with
|
||||
`{"type": "memory_config", ...}`: `chat` (true), `others` (true), `off` (false), `since_days`
|
||||
(null: whole history), `recent_minutes` (30), `recent_records` (20), `matches` (5),
|
||||
|
||||
Reference in New Issue
Block a user