Commit Graph

40 Commits

Author SHA1 Message Date
javis-bot
cbe898d3d0 feat(app): log screen saves (shown lines, conversation, voice recordings), stop-or-finish on a new request, video pages that play
- 로그 screen: "보이는 글 txt로 저장" (logs_save_text), a 대화 view that saves the days'
  conversation with every trace in full (history_export), and a 목소리 view with the
  voice.record switch and the days' recordings as a zip with an index (voice_export).
- Voice worker: with JARVIS_RECORDINGS_DIR (voice.record, off by default) each final and each
  spoken answer is kept as a WAV with an index line; day folders older than 7 days removed.
- response.interrupt (default on): off, latest/important finish the running answer first.
- Bot browser: Chrome lets pages start video and sound; botBrowser reports each page's media
  state, waits for a player to start after open/click, and presses play on a video page a click
  led to when its player stayed paused (live: a clicked YouTube video stayed paused).
- webSearch reads `query` when `search_query` is missing (live: both searches of an answer failed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 15:27:57 +09:00
javis-bot
6fc3ad3d57 feat(worker): answer traces stored from the question on and filled in as the answer is made
A trace is now kept the moment the request arrives (at the time it was
asked, ms -1), its steps are stored as the engine adds each one, and it
ends with the answer. Traces a killed run left unfinished are closed at
start. The 대화 screen reads a trace still being made as pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:03:45 +09:00
javis-bot
46f71e7f03 feat(app): 대화 screen to read what people said, Jarvis's answers and how each answer was made
The management app reads the worker's history store (read only) and shows
people's words, Jarvis's answers and the answer traces in time order, with
filters for kind, place, person, channel, day and text, live follow, older
items on demand and expandable traces (records given, tools offered, plan,
model thinking, tool calls and results).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 03:08:38 +09:00
javis-bot
a20ebe4eae feat(brain): keep how each answer was made (records, tools, plan, thinking, tool calls)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 03:00:10 +09:00
javis-bot
27eb1315b9 fix(tts): MeloTTS speaks up to 1.25 itself so fast speech sounds natural
Stretching all of the speed-up from 1.0 kept every sound but sounded
awkward (UTMOS 3.13 at 1.5, against 3.73 for MeloTTS's own 1.5), while
MeloTTS's own speed past 1.25 drops sounds (sentence-final "요" and first
syllables, 4.6% character errors at 1.5). MeloTTS now speaks up to 1.25
and the WSOLA stretch does only the rest, with a 5 ms search: 2.6% and
UTMOS 3.56 at 1.5 through the real child. Version 0.1.6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 00:06:52 +09:00
javis-bot
dbe5c617bc feat(tts): change the speech speed without a restart and keep fast speech clear
MeloTTS's own speed shortens every sound and slurred fast speech (Whisper
large-v3 on five Korean sentences, three renders each: 5.5% character
errors at 1.5 and 20% at 2.0, against 1.4% at 1.0). The MeloTTS child now
speaks at most at its natural pace and makes faster speech with a WSOLA
time stretch that keeps the pitch (1.9% at 1.5, 2.2% at 2.0, about 15 ms
per 4 s of speech).

A voice.tts.speed saved while the services run goes to the worker at once
(new tts_speed message) and applies from the next sentence; the client
sends it again whenever the worker connects again. Version 0.1.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 23:39:04 +09:00
javis-bot
53bccb620d feat: screen viewing, Jarvis sees the whole screen, a monitor or a window for registered people
- screenView tool (screen_view.spec.md): whole screen, monitor by number, window by part of its
  title or program, or the list of monitors and windows; fast, normal or detailed pictures.
- The app captures with Windows Graphics Capture (app/src/screen.rs, admin.spec.md "Screen
  viewing"), decides for the asker (only people registered for PC control), never picks, lists
  or shows excluded programs (pcControl.screenExclude, password managers by default), refuses
  minimised windows and black pictures, and records screen.view in the activity log.
- Pictures ride on the tool message (ToolExecutionResult.images); llm.py sends them to
  OpenAI-compatible servers as image parts after the tool messages. They stay in that reply only:
  not in the tool carryover, logs or prompt dumps.
- The LLM reads pictures only when screen viewing is available (--vision gpu normally, cpu in a
  game), measured apart for the placement; otherwise screenView is a disabled tool.
- Worker: screen_action / screen_action_result, 4 MiB messages.
- Eval evals/test_screen_view_tool.py; Windows check tests/screen_windows.rs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 17:02:59 +09:00
javis-bot
324fd2ea0a feat: news notifications for the background collection and a collectedNews tool
Each collection target can announce new items at once or in a daily digest
(off by default) in collect.notifyChannel, as titles and links only, with no
LLM; each post is recorded in the activity log. The brain gets a read-only
collectedNews tool (every target's newest items by default, fenced as
untrusted web content), offered with the app's collection file and the
collection feature, and available to scheduled tasks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:22:54 +09:00
javis-bot
a5bd25b86f feat: personal notes, each person's standing instructions for how Jarvis answers them
A personalNotes tool (add, list, delete the asker's own notes) offered only
with a notes backend. The app keeps the notes in config/notes.json and sends
the asker's with every voice, chat and task reply; the reply engine puts them
in a <person_notes> fence as that person's preferences, which never permit an
action or a tool. The worker's schedule bridge becomes a shared app bridge
for schedule_action and note_action. A 사람별 지침 screen edits and deletes
any note; features.notes turns the tool and the notes off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:10:53 +09:00
javis-bot
04b603facd feat(app): an activity log of what Jarvis did (chat actions, schedule deliveries)
Each chat channel action the brain's tool asked for and each schedule
delivery is recorded in activity/<day>.jsonl by kind, target, who asked and
outcome, never its text, kept 30 days. The worker's chat_action carries the
asker. A 한 일 screen lists the newest entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:51:18 +09:00
javis-bot
ea2d3ba958 feat: reminders and scheduled tasks (schedule tool, app store and delivery)
The brain gets a schedule tool (add, list, delete the asker's own reminders,
timers and tasks) offered only with a schedule backend. The voice worker asks
the app through schedule_action and runs fired tasks as reply via task with
read-only tools, no MCP and a fresh short-term memory. The app keeps the
schedules in config/schedules.json, fires them every 5 s, speaks them when the
person is in Jarvis's voice channel and otherwise posts in the chat channel
they asked in (or schedule.chatChannel for voice). A feature switch and a
예약 screen list and delete everyone's schedules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:27:42 +09:00
javis-bot
3d58cf8ab1 feat(voice): in a game, partial transcription waits while an answer is written
- the app's ready message says whether the game stage runs (llm ready game=true); the worker then
  holds partial passes while a reply is being written and runs them again, the newest audio per
  speaker, when no answer is; finals never wait; without a game nothing changes
- M0 gamemode_tune decision: the game placement answered 1.7 times faster with partials held
  (64.9 against 37.5 tok/s with a partial pass every 2 s)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:58:11 +09:00
javis-bot
aa6d950af6 fix(app): performance stage switches survive failures, reconnects and late signals
- the normal stage reads other programs' memory afresh (the card after the LLM stopped, less the
  STT worker's model) and keeps it; the game stage counts from the latest normal reading, so the
  game is not counted against its own reserve
- a restart that fails brings back the stage that ran before and tells the brain ready; the
  failed stage is tried again after 60 s, three times at most; when nothing comes back the brain
  is told paused so requests fail at once
- the worker client keeps the latest llm state and sends it first on every connection
- switches are numbered: llm switching carries an id, llm_idle names it, and a late idle of an
  earlier switch no longer lets a new one through
- a request that waited 180 s for a switch fails without calling the brain
- the dashboard shows the stage that runs also on a fallback placement without experts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:34:24 +09:00
javis-bot
7e096b449e feat(app): performance stages move the LLM between the normal and the game placement
- while the services run, game mode decides the stage (perf.rs): a game means the game stage at
  once (or pause with game.pauseInGame); its end brings the normal stage back after
  game.revertSeconds (default 30)
- a change tells the voice worker the LLM switches (llm switching, with a notice), waits until
  the brain has written the answers in progress (llm_idle, at most 2 minutes), restarts the LLM in
  the stage and tells the worker it is ready or paused
- the game stage runs at 16K context, one request at a time, half the engine's CPU threads and
  the game placement (reserve game.reserveGb, default 8, else 2 GiB, else 0 GiB, else pause),
  measured once at that context; every placement of a run counts other programs' memory as it
  was when the services started
- the voice worker gates every LLM use: on switching new work waits (at most 180 s), the first
  spoken request that waits hears the notice once under its own id, on ready all go on, on
  paused nothing is answered
- dashboard: "성능 단계: 게임 중 · 게임에 X GB" / 일시정지; game mode screen: game VRAM, revert
  wait, pause in game

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:23:21 +09:00
javis-bot
55d19364a0 fix(app): the proxy refuses IPv6 outside global unicast and its stop closes open tunnels
- IPv6 counts as public only in 2000::/3 (without 2001::/23, documentation and 6to4), so site
  local fec0::/10 and IPv4-compatible addresses are refused; the collector uses the same rule
- every proxy connection is a task of one set: a stop ends the open CONNECT tunnels and waits
  until their sockets are gone

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 22:07:50 +09:00
javis-bot
69f1bf4fa9 fix(app): collection fetches addresses with Korean paths and words
The request line is ASCII: other characters and spaces in the path and query go as UTF-8 %HH,
while existing %HH and separators stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:49:01 +09:00
javis-bot
d0cb540fa0 fix(app): collection's 15 s limit also covers the name lookup, TLS and headers
The name lookup runs on a daemon thread waited for only until the deadline, and a watchdog
shuts the connection down at the deadline, so a server that sends its status line or headers
byte by byte within the per-read limit is cut off too. A response cut short is a failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:46:10 +09:00
javis-bot
9185265010 fix(app): collection connects only to the addresses it checked, within 15 s, logging no address
- the fetcher resolves once, requires every address to be public and connects only to those
  (TLS still checks the name), so DNS rebinding cannot reach the local network; redirects too
- 15 s for the whole fetch also against servers that send slowly; no URL in any log line
- a site whose title alone changed is kept again
- target names that differ only in case are refused (one Windows folder)
- the worker always gets the collection file, so results older than the keep days go also
  with no targets or the feature off
- the results screen lists every target with results, not only those in the newest 100

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:39:04 +09:00
javis-bot
b94e9dc112 feat(app): background collection frame for sites, search words and RSS feeds
Targets (none by default), interval and keep days on a new 백그라운드 수집 screen with the
results; the worker collects on its own thread through the web search's address check and
writes only to the data folder's collect/ (logs carry target names and counts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:29:49 +09:00
javis-bot
8b3905f1c2 fix(app): MCP discovery never blocks a turn; secrets masked as lines are written; TTS child gets no MCP secrets
- Discovery runs outside the MCP tool cache's lock (one at a time), so a turn reading the cached
  tools never waits for a slow server; a new conversation refreshes in the background.
- The log hub masks secrets before a line is kept or written, so a key replaced or removed later
  stays hidden on the screen, in copies, files and exports, also after a restart.
- The worker takes JARVIS_MCP_ENV out of its environment at start, before MeloTTS runs, and hands
  it to the brain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 10:07:52 +09:00
javis-bot
3ef4281180 feat(app): features screen with tool switches and MCP servers; v2 drops host file and screen tools
- Settings `features` (weather, webSearch, meals) and `mcpServers` (name, command, args, env names,
  enabled), edited on the new 기능 screen. The tools of a feature that is off go into the brain's
  `disabled_tools`; only enabled MCP servers go into its `mcps`.
- MCP environment values (API keys) live only in the encrypted secrets (`mcp:<server>:<NAME>`),
  shown as stored or not, masked in logs, removed with their server, and reach the worker only in
  `JARVIS_MCP_ENV`, which the brain takes out of its environment before giving each server its own.
- `enabled_builtin_tools(cfg)` is the one list the router, toolSearchTool and the executor use, so
  a turned-off tool is neither offered nor run, also when the model names it.
- The v2 brain never offers v1's `localFiles` (home folder read/write/delete) and `screenshot`:
  anyone in the voice or chat channels could ask for them. Screen viewing comes in M7.
- The brain connects MCP servers on its own thread and logs each outcome without debug logging.
- Stored secrets are masked in the log hub as soon as the data folder is found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 09:56:17 +09:00
javis-bot
a4436698c8 feat(app): M5 step 4a, install the runtime and voice models into the data folder
- download: resumed downloads checked against a pinned SHA-256.
- install: uv 0.11.20, the worker/brain venv (Python 3.12, the new
  src/voice_worker/requirements-brain.txt), the MeloTTS venv (Python 3.11,
  PyTorch cu128), Whisper large-v3-turbo, the Korean MeloTTS voice and
  Strata, the M0 kit's recipe; markers per step and version, cancel, retry,
  everything (caches, temp, HF and NLTK data) inside the data folder, the uv
  cache cleaned and the worker settings pointed at the venvs when done.
- The services give the worker the same environment once installed.
- The wizard's model step shows and drives the install.

Installed from an empty folder on Linux and started the services on it:
worker connected, STT and TTS ready, nothing written outside the folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:11:23 +09:00
javis-bot
eb0fa2f159 fix(app): log hub masks dotted tokens, keeps worker errors and drops the oldest lines by time
- redact: a token with sentence dots around it is masked too.
- Worker error messages are always logged (code and id), so the log screen
  and the error report show them without debug logging.
- The voice worker logs with stt, llm and tts categories, so its lines land
  in those tabs.
- The zip leaves out the oldest lines by the time they were logged, across
  every tab's file, not by file name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:43:01 +09:00
javis-bot
46e4e75d05 feat(voice_worker): a short voice prompt and one judgement before each v2 answer
- compact_prompt: the v2 persona in a tenth of v1's words, answers as short as
  possible, the full tool guidance only on turns with a data tool. A greeting's
  reply prompt went from about 4,500 to 1,400 tokens on Strata.
- intent_judge: the tool router judges whether the turn needs tools; the
  planner runs only when it does. prepare_intent runs the pick ahead on a
  partial transcript (worker message prepare), and the reply reuses it.
- Without stream_browser the screen-share tools are not offered.
- The time context carries the 12-hour form (10:37 was read as pm).
- A <tool_call> block written as the reply is parsed as a call, not spoken.

Evals (Qwen3.6 on Strata, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1):
221 passed, 4 failed (known failures); new evals/test_voice_reply_length.py
(v2 answers 234 chars against v1's 416) and test_reply_language_default.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 12:25:15 +09:00
javis-bot
7623d6d002 feat(voice_worker): the v2 brain answers in Korean unless asked for another language
The brain passes default_language (worker --reply-language, default Korean) to
the reply engine; unlike the output_language lock it gives way when the user
explicitly asks for a language. Tools get the ko code (--tool-language).

Eval evals/test_reply_language_default.py (Qwen3.6 on Strata): 6/6, English
questions answered in Korean, an explicit English request in English, and the
control without the default answers English in English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:34:12 +09:00
javis-bot
9ff307d7fe fix(voice_worker): the v2 brain answers in the asker's language
v1's tts_engine default (English-only Piper) made the reply engine force
English, so Korean questions got English answers. The v2 brain speaks
through the worker's MeloTTS and chat answers are not spoken, so it runs
with tts_engine=melo; an output_language lock still wins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:16:34 +09:00
javis-bot
9e75df266b fix: a delete question counts as asked only once the answer reached the person
A person's reply stopped counting as "unanswered" as soon as the worker sent
reply_text, so a delete question that could not be spoken (no TTS, a failed
or stopped speech) or that the app could not post still let the next
"응, 지워" confirm the delete.

- Worker: an answer reaches the person on a tts_done neither failed nor
  cancelled (spoken answers), on the app's reply_delivered ok (chat
  answers), or at reply_text (unspoken voice answers). tts_unavailable, a
  failed or cancelled tts_done, reply_delivered ok=false, no report within
  30 s and the session ending all mean it never did: a turn number is
  skipped, so the delete asked in it lapses. A reply sent before the
  previous answer reached them is still not their next turn.
- App: a chat answer reports reply_delivered with whether Discord took the
  post; a late, failed or empty answer reports it was not posted.
- Tests: no TTS, a failed speech, a chat post the app reports failed, and
  no report at all do not delete; a spoken and a posted question do.
  Conversation and worker client tests for the report.
- Specs: voice_worker.spec.md (reply_delivered, when an answer reaches the
  person), discord_core.spec.md, conversation.spec.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:39:48 +09:00
javis-bot
c328bda699 fix(voice_worker): a yes sent before the delete question was answered does not delete
A delete is confirmed only in the person's very next turn, but turn numbers
were given when a reply started. A "응, 지워" sent while the delete question
was still being written waited for the person's lock and then counted as the
next turn, so the delete went through.

- The worker counts each person's unanswered replies. A reply that arrived
  before their previous answer was given skips a turn number, so a delete
  asked in that answer lapses.
- After an answer the person never got (failed, or stopped before it was
  spoken) a turn number is skipped as well: it asked them nothing.
- Tests with the real chatChannels tool and an agreeing consent check: a yes
  sent before the answer does not delete, a yes after a failed answer does
  not delete, a yes after the answer deletes.
- Specs: voice_worker.spec.md (turn numbering), chat_channels.spec.md (the
  next turn is a message sent after the question was received).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:32:25 +09:00
javis-bot
19f984ae73 fix(voice_worker): one person's voice and chat answers are written one at a time
The voice and chat lanes let a person's voice reply start while their chat
reply was still running. The chat delete consent counts "the very next
message" by turn number, so a message that started before the delete was
asked could pass as the confirmation, and the person's dialogue memory
was updated out of order.

- A reply takes its asker's lock, then its lane's: one person's replies
  stay one at a time in the order asked, different people's voice and
  chat answers still overlap, and a reply waiting for its asker holds no
  lane. Test: a person's voice question starts only after their chat
  answer.
- Specs and roadmap: the lanes are per person; the roadmap no longer says
  voice answers wait behind chat answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:27:13 +09:00
javis-bot
2f69557e1b fix: voice answers no longer wait behind chat answers; late chat answers are dropped
- The voice worker writes voice and chat replies in separate lanes (one
  of each at a time), so a chat answer being written never holds up a
  voice answer. Tests: a voice reply finishes while a chat reply is still
  running; replies of one kind stay one at a time.
- A chat answer arriving 5 minutes or more after it was asked is not
  posted, checked when it arrives.
- Specs: the lanes, and that Strata needs --parallel 2 for both lanes'
  LLM calls to run at once until the app starts Strata itself (M5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:42:30 +09:00
javis-bot
f892a1dce9 feat: answer calls in the voice channel through the response policy
- conversation.rs: the loop between the core's events and the rest.
  Finals go through call detection and the response policy; answers are
  asked of the brain and spoken from the app's queue, an interruption
  cancels only that answer, a voice stop silences everything, and missed
  requests are kept without text (last 100). Tested with a fake voice side.
- Core: answer(user, text) adopts a playback request the worker speaks
  under the same id, cancel(id) stops one request, ask_addressed and
  ask_rank pass the conversation's questions; new SpeakingStopped,
  Addressed and Ranked events.
- Worker client: reply, addressed and rank messages and their results; a
  reply with no answer, refused or not sent ends its speech as failed.
- Voice worker: addressed and rank messages on their own threads, beside
  a running reply; a spoken reply cancelled while the brain is answering
  is not spoken. judges.py holds the two short LLM judgements with the
  heard text fenced as data; eval evals/test_voice_judges.py.
- Specs (conversation, discord core, voice worker), docs/llm_contexts.md
  and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:26:47 +09:00
javis-bot
0182823592 fix(chat): delete needs the person's own yes; chat action ids never repeat
- chatChannels delete with confirm=true now runs only in the person's very
  next turn and only when their own message of that turn agrees, checked by a
  small fail-closed YES/NO call on the router model chain (the message fenced
  as data). A refusal, question, other topic or error cancels the request;
  an unconfirmed request lapses after one turn. The reply model's confirm
  flag alone can no longer delete.
- chat_action ids carry a per-run nonce and the session, so a late answer
  meant for an earlier connection (or worker run) never completes a newer
  action.
- Tests: agree, refuse, other topic, lapse, classifier failure, late answer
  across reconnects. Eval test_chat_delete_consent: 12 of 12 on qwen2.5:7b,
  3 runs. Spec and llm_contexts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:08:35 +09:00
javis-bot
fe4e296361 feat(chat): chatChannels tool for the registered Discord chat channels
- Tool (chat_channels.spec.md): list, read, search, send, edit, react,
  delete, pin, unpin. Offered only when the v2 brain gives the turn a chat
  backend, and then on every turn. Channel by ID, name, part of a name or a
  name inside the text; a message ID finds its channel; planner forms of the
  action ("send_message") work. Read and search results are fenced as
  untrusted data. delete only reports until called again with confirm=true
  in a later turn of the same person.
- Worker: chat_channels from the app, ChatBridge reads/searches the history
  store and sends chat_action, waiting for chat_action_result (15 s timeout,
  failed on session close); per-person reply turns and pending deletes.
- App: channel switches edit/react (on) and delete/pin (off); chat_channels
  sent on worker ready and after login; chat_action runs send, edit (own
  messages only), react, delete, pin, unpin and answers with ids or the error.
- Engine: chatChannels left out of the catalogue without a backend; the
  "router returned everything" check now tolerates that.
- live_discord: edit own, refuse editing the speaker's, react, pin (Discord
  refuses in a voice channel's chat, reported as an error), delete, refuse an
  unregistered channel; passed in 109.4 s.
- Eval (qwen2.5:7b, 3 runs): 10 of 12; never obeyed the "delete everything"
  message and never deleted in the same turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:01:33 +09:00
javis-bot
3dbf1e8c56 feat(nutrition): meals belong to the person who logged them
- meals.user_id (v1 databases gain it on open, DEFAULT '' keeps their meals
  the single user's); insert_meal, get_meals_between and delete_meal take
  the person, and delete only removes that person's meal.
- turn_state.set_user / ToolContext.user_id carry the asker; logMeal,
  fetchMeals and deleteMeal use it. The v2 brain sets it for each reply.
- Tests: two people never see or delete each other's meals, no user is v1's
  single user, and a v1 database opens with its meals kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:32:36 +09:00
javis-bot
b3dcd12ec9 fix(voice_worker): keep history and the brain's output scoped to who was there
- A final is stored under the channel, speaker name and time fixed when its
  end was decided. Decoding takes a while; a move, rename or late joiner
  meanwhile used to change where and when it was said, and who heard it.
- At start the worker closes presence a killed run left open
  (HistoryStore.close_open_presence), so a person who is gone no longer
  hears new speech after a restart.
- run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm
  profile, extractor, diary, graph, digest); the v2 brain uses it so another
  user's profile or diary never reaches the prompt.
- While a brain reply runs, stdout and stderr are muted (also helper threads
  such as the web fetch pool); the worker logs through voice_worker/log.py to
  the original stderr.
- Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the
  prompt in earlier runs; with an 8k tag the chat-record question was answered
  4 of 4 and nothing reached the output. Noted in llm_contexts and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:28:25 +09:00
javis-bot
864b2e7fdb feat(voice_worker): record history from the app and answer reply requests with the brain
Worker (voice_worker.spec.md "History and brain"):
- --history <file> keeps the conversation history store; --brain hosts the
  jarvis reply engine (JarvisBrain, settings point ollama_base_url at /v1).
- voice_join / voice_leave / member_join / member_leave keep presence; the
  session end closes it. Non-empty finals are stored under the speaker's name
  in the current voice channel.
- chat_message / chat_edit / chat_delete keep registered chat channels.
- reply builds the asker's scoped history context, runs the brain on a thread
  (one at a time), answers reply_text, stores a voice answer, and with
  speak=true says it under the same id. memory settings in hello and
  memory_config. brain_unavailable without --brain.

App (discord_core.spec.md "History reporting"):
- WorkerClient sends the new messages; the core reports the voice channel
  and its heard members after a join, a move and every worker reconnect,
  member joins and leaves, voice_leave when a session ends, and every message
  of a readable registered channel (its own too, with `to` on replies), edits
  and deletes.
- live_discord runs the worker with --history and checks the store at the
  end: 8 finals under the speaker's name, the edited chat message only,
  Jarvis's own messages, 5 presence intervals, none left open (passed, 107.8 s).

Eval: history recall gains a chat-record case and fails a reply that reads a
record stamp out (field run with qwen2.5:7b did once in four).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:16:24 +09:00
javis-bot
5838f9edd8 feat(voice_worker): free a speaker's place when they leave the voice channel
- New speaker_leave message: ends the speaker's open utterance (its final is still sent) and drops
  their state, so a long session never hits the 32-speaker limit as people come and go.
- Spec and test cover more than 32 speakers in one session, each leaving mid-utterance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:14:25 +09:00
javis-bot
1309525aa8 fix(voice_worker): the wake word hint may never add, drop or create a wake word
Hotword occurrences were counted with fuzzy matching, which never counted a one-letter wake word
(so a hinted "봇 ..." was accepted over a plain decode without it) and counted the real word
"서비스" as "자비스" (so the hint could turn it into the wake word). Occurrences are now counted
exactly (case, spacing and punctuation ignored); near misses are left as heard for call detection.
The real-noise eval now plays each recording as recorded and at -25 dBFS and reports how many clips
reached the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:40:42 +09:00
javis-bot
590cd07268 fix(voice_worker): even out speaker loudness and stop transcribing noise into words
- Loudness equalisation replaces peak normalisation: rumble below 60 Hz removed, level measured on
  the VAD-voiced windows only, brought to -20 dBFS (gain -20..+40 dB), soft limiter below 0.98.
- Only clearly voiced windows (VAD >= 0.5) count towards the 250 ms needed before an utterance
  reaches the model, so noise the VAD half-believes never calls Whisper.
- Hallucination gate: mean token log-probability >= -0.7 (the old len/(len+1) correction let short
  hallucinations through), compression ratio <= 2.4; no_speech_prob dropped (always 0.0 on turbo).
- With hotwords, the plain decode of the same encoder pass decides; the hint may fix a wake word's
  spelling but cannot add or drop one (it wrote the wake word on laughter and after real sentences).
- Opt-in real-model evals: voice-like babble with the hint, speech over babble, quiet and clipped
  speakers, and a JARVIS_NOISE_DIR folder of real noise recordings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:27:38 +09:00
javis-bot
a7685e9078 feat(voice_worker): M1 multi-speaker streaming STT and sentence TTS over a local WebSocket
- One shared faster-whisper model for every speaker, batched; finals go before partials and
  partial batches are capped at two so a final never waits behind a ten-speaker pass.
- Per-speaker 48k stereo to 16k mono conversion, streaming Silero VAD, utterance ends on VAD
  silence, Discord speaking_end, missing audio, or the 28 s Whisper window.
- MeloTTS in its own venv through a child process; sentences stream out one chunk each, with
  incremental text, cancel and failure reporting.
- Loopback-only server with a token from the environment; logs carry no transcript text.
- Spec in src/voice_worker/voice_worker.spec.md; unit tests with fake engines and an opt-in
  real-GPU test that streams ten speakers at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:00:06 +09:00