Commit Graph

73 Commits

Author SHA1 Message Date
javis-bot
4621e6c1fd fix(chat): the delete consent check knows the pending message and reads the whole reply
The consent classifier is now told which message and channel are pending, so a yes for another
message ("yes, delete message 22" while 21 is pending) is a no and cancels the request. A reply
longer than 600 characters is no longer cut and judged by its start (which could drop a later
"don't"); it counts as no without a model call.

Eval evals/test_chat_delete_consent.py gains the different-message and long-reply cases:
17/17 on qwen2.5:7b, three runs. The previous prompt said YES 3/3 to
"응, 일정 채널에 있는 거 지워" while a message in 잡담 was pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:26:58 +09:00
javis-bot
0182823592 fix(chat): delete needs the person's own yes; chat action ids never repeat
- chatChannels delete with confirm=true now runs only in the person's very
  next turn and only when their own message of that turn agrees, checked by a
  small fail-closed YES/NO call on the router model chain (the message fenced
  as data). A refusal, question, other topic or error cancels the request;
  an unconfirmed request lapses after one turn. The reply model's confirm
  flag alone can no longer delete.
- chat_action ids carry a per-run nonce and the session, so a late answer
  meant for an earlier connection (or worker run) never completes a newer
  action.
- Tests: agree, refuse, other topic, lapse, classifier failure, late answer
  across reconnects. Eval test_chat_delete_consent: 12 of 12 on qwen2.5:7b,
  3 runs. Spec and llm_contexts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:08:35 +09:00
javis-bot
fe4e296361 feat(chat): chatChannels tool for the registered Discord chat channels
- Tool (chat_channels.spec.md): list, read, search, send, edit, react,
  delete, pin, unpin. Offered only when the v2 brain gives the turn a chat
  backend, and then on every turn. Channel by ID, name, part of a name or a
  name inside the text; a message ID finds its channel; planner forms of the
  action ("send_message") work. Read and search results are fenced as
  untrusted data. delete only reports until called again with confirm=true
  in a later turn of the same person.
- Worker: chat_channels from the app, ChatBridge reads/searches the history
  store and sends chat_action, waiting for chat_action_result (15 s timeout,
  failed on session close); per-person reply turns and pending deletes.
- App: channel switches edit/react (on) and delete/pin (off); chat_channels
  sent on worker ready and after login; chat_action runs send, edit (own
  messages only), react, delete, pin, unpin and answers with ids or the error.
- Engine: chatChannels left out of the catalogue without a backend; the
  "router returned everything" check now tolerates that.
- live_discord: edit own, refuse editing the speaker's, react, pin (Discord
  refuses in a voice channel's chat, reported as an error), delete, refuse an
  unregistered channel; passed in 109.4 s.
- Eval (qwen2.5:7b, 3 runs): 10 of 12; never obeyed the "delete everything"
  message and never deleted in the same turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:01:33 +09:00
javis-bot
3dbf1e8c56 feat(nutrition): meals belong to the person who logged them
- meals.user_id (v1 databases gain it on open, DEFAULT '' keeps their meals
  the single user's); insert_meal, get_meals_between and delete_meal take
  the person, and delete only removes that person's meal.
- turn_state.set_user / ToolContext.user_id carry the asker; logMeal,
  fetchMeals and deleteMeal use it. The v2 brain sets it for each reply.
- Tests: two people never see or delete each other's meals, no user is v1's
  single user, and a v1 database opens with its meals kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:32:36 +09:00
javis-bot
b3dcd12ec9 fix(voice_worker): keep history and the brain's output scoped to who was there
- A final is stored under the channel, speaker name and time fixed when its
  end was decided. Decoding takes a while; a move, rename or late joiner
  meanwhile used to change where and when it was said, and who heard it.
- At start the worker closes presence a killed run left open
  (HistoryStore.close_open_presence), so a person who is gone no longer
  hears new speech after a restart.
- run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm
  profile, extractor, diary, graph, digest); the v2 brain uses it so another
  user's profile or diary never reaches the prompt.
- While a brain reply runs, stdout and stderr are muted (also helper threads
  such as the web fetch pool); the worker logs through voice_worker/log.py to
  the original stderr.
- Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the
  prompt in earlier runs; with an 8k tag the chat-record question was answered
  4 of 4 and nothing reached the output. Noted in llm_contexts and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:28:25 +09:00
javis-bot
864b2e7fdb feat(voice_worker): record history from the app and answer reply requests with the brain
Worker (voice_worker.spec.md "History and brain"):
- --history <file> keeps the conversation history store; --brain hosts the
  jarvis reply engine (JarvisBrain, settings point ollama_base_url at /v1).
- voice_join / voice_leave / member_join / member_leave keep presence; the
  session end closes it. Non-empty finals are stored under the speaker's name
  in the current voice channel.
- chat_message / chat_edit / chat_delete keep registered chat channels.
- reply builds the asker's scoped history context, runs the brain on a thread
  (one at a time), answers reply_text, stores a voice answer, and with
  speak=true says it under the same id. memory settings in hello and
  memory_config. brain_unavailable without --brain.

App (discord_core.spec.md "History reporting"):
- WorkerClient sends the new messages; the core reports the voice channel
  and its heard members after a join, a move and every worker reconnect,
  member joins and leaves, voice_leave when a session ends, and every message
  of a readable registered channel (its own too, with `to` on replies), edits
  and deletes.
- live_discord runs the worker with --history and checks the store at the
  end: 8 finals under the speaker's name, the edited chat message only,
  Jarvis's own messages, 5 presence intervals, none left open (passed, 107.8 s).

Eval: history recall gains a chat-record case and fails a reply that reads a
record stamp out (field run with qwen2.5:7b did once in four).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:16:24 +09:00
javis-bot
7a288129de fix(memory): rank the whole history and keep relevant recent records
- Search scored only the newest 5,000 matching records, so an old record
  behind enough newer chatter was never found. Both passes (word rarity counts
  and the score) now run in SQLite over every record in scope and only the
  best rows come back.
- context() excluded the recent window from search and then cut that window
  for the budget, so a relevant recent record could vanish. Search now covers
  the recent window, matches keep their half of the budget, and a matched
  record is removed from the recent block so it appears once.
- Matching used SQLite's ASCII-only lower() against Python's lowercasing.
  Stored text gets a folded column (NFKC + casefold) and query words are
  folded the same way, so case is ignored in every script.
- Prompt label "Records related to this question" (matches may now be recent);
  eval re-run 3x on qwen2.5:7b, all pass. Spec, memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:50:37 +09:00
javis-bot
54a465948a feat(memory): v2 conversation history store and its context in the reply prompt
- src/jarvis/memory/history.py keeps voice transcripts, Jarvis's answers,
  registered chat channel messages (with edits and deletes) and voice channel
  presence in one SQLite file. Every read is scoped to the asker: own records
  always, chat channels and others' words (only while the asker was in the
  same voice channel) behind settings, plus an optional since and off.
- Search is language-agnostic: Unicode words, prefixes keeping 60% of the
  word, rarer words weigh more, newer wins ties. 100k records search in
  24-61 ms here. context() returns the recent window and the matches cut to a
  token budget, matches at most half so chatter cannot push out the record
  the question is about.
- run_reply_engine(history=...) puts that context into the system prompt
  under "Information shared in previous conversations", fenced as data.
  evals/test_history_context_recall.py: with the block qwen2.5:7b answers the
  stored day in Korean and English (3 of 3 runs), without it the day is unknown.
- Spec history.spec.md, registry entry in CLAUDE.md, reply.spec.md,
  docs/llm_contexts.md, docs/v2/memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:45:33 +09:00
javis-bot
6b23a8adc7 feat(llm): talk to OpenAI-compatible servers such as Strata
A base URL that ends in /v1 now selects the OpenAI-compatible API: chat goes
to /chat/completions and embeddings to /embeddings. The conversion lives in
src/jarvis/llm.py only, so the reply engine and every other caller keep their
Ollama-shaped messages and responses.

- Requests drop private engine keys, send tool-call arguments as JSON strings,
  map num_predict to max_tokens and the thinking switch to
  chat_template_kwargs.enable_thinking, and leave out num_ctx and keep_alive.
- Responses come back with message.content, tool calls with dict arguments and
  an id, and message.thinking from reasoning_content.
- Streaming, the warm-up (one-token chat completion), the intent judge and the
  dictation clean-up use the same path. A server without an embedding model
  fails open to the text paths.
- Specs and docs/llm_contexts.md describe the transport; roadmap records M3
  step 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:32:48 +09:00
javis-bot
5838f9edd8 feat(voice_worker): free a speaker's place when they leave the voice channel
- New speaker_leave message: ends the speaker's open utterance (its final is still sent) and drops
  their state, so a long session never hits the 32-speaker limit as people come and go.
- Spec and test cover more than 32 speakers in one session, each leaving mid-utterance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:14:25 +09:00
javis-bot
1309525aa8 fix(voice_worker): the wake word hint may never add, drop or create a wake word
Hotword occurrences were counted with fuzzy matching, which never counted a one-letter wake word
(so a hinted "봇 ..." was accepted over a plain decode without it) and counted the real word
"서비스" as "자비스" (so the hint could turn it into the wake word). Occurrences are now counted
exactly (case, spacing and punctuation ignored); near misses are left as heard for call detection.
The real-noise eval now plays each recording as recorded and at -25 dBFS and reports how many clips
reached the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:40:42 +09:00
javis-bot
590cd07268 fix(voice_worker): even out speaker loudness and stop transcribing noise into words
- Loudness equalisation replaces peak normalisation: rumble below 60 Hz removed, level measured on
  the VAD-voiced windows only, brought to -20 dBFS (gain -20..+40 dB), soft limiter below 0.98.
- Only clearly voiced windows (VAD >= 0.5) count towards the 250 ms needed before an utterance
  reaches the model, so noise the VAD half-believes never calls Whisper.
- Hallucination gate: mean token log-probability >= -0.7 (the old len/(len+1) correction let short
  hallucinations through), compression ratio <= 2.4; no_speech_prob dropped (always 0.0 on turbo).
- With hotwords, the plain decode of the same encoder pass decides; the hint may fix a wake word's
  spelling but cannot add or drop one (it wrote the wake word on laughter and after real sentences).
- Opt-in real-model evals: voice-like babble with the hint, speech over babble, quiet and clipped
  speakers, and a JARVIS_NOISE_DIR folder of real noise recordings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:27:38 +09:00
javis-bot
a7685e9078 feat(voice_worker): M1 multi-speaker streaming STT and sentence TTS over a local WebSocket
- One shared faster-whisper model for every speaker, batched; finals go before partials and
  partial batches are capped at two so a final never waits behind a ten-speaker pass.
- Per-speaker 48k stereo to 16k mono conversion, streaming Silero VAD, utterance ends on VAD
  silence, Discord speaking_end, missing audio, or the 28 s Whisper window.
- MeloTTS in its own venv through a child process; sentences stream out one chunk each, with
  incremental text, cancel and failure reporting.
- Loopback-only server with a token from the environment; logs carry no transcript text.
- Spec in src/voice_worker/voice_worker.spec.md; unit tests with fake engines and an opt-in
  real-GPU test that streams ten speakers at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:00:06 +09:00
javis-bot
8f2470af2a fix(m0): keep the game in front for the whole frames window and record the game_frames results
On the game PC the game went behind another window right after Jarvis
started and stayed there for three windows, drawing at its background rate
(about 58 fps). The kit only checked the window in front at the end of a
window, so the window in which the user clicked back into the game counted
and showed yielding as a 47 % drop in 1 % low.

- FrontWatch checks every second from the start of a window to its end and
  spoils the window if the game was behind anything at any check.
- Before a window the kit asks the user, with a beep, to go back to the
  game and waits up to two minutes.
- What was in front is kept only as a category (a Jarvis process, a console
  window, Explorer, another app), never an app name.

Results: with the game in front, a busy Jarvis cost 0.2 % of the average
fps and 3.3 % of the 1 % low with no frames over 50 ms, so by the
pre-registered rule nothing is added and yielding is not adopted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:35:38 +09:00
javis-bot
cec52165b2 fix(m0): judge game frames by what the screen showed and drop spoilt windows
- Frame rate, 1 % low and hitches come from msBetweenDisplayChange of the
  frames that reached the screen; frames never shown are counted apart and
  the presented rate is kept as presented_fps. A game presenting 60 fps
  while the screen changes every 167 ms now shows as the stutter it is.
- A window whose capture failed, a busy window where the LLM finished no
  answer, and every busy window when the STT load failed are not counted.
- A yielding window counts only when every Jarvis process got both its
  CPU and its GPU priority lowered.
- The "game hardly drawing" floor applies only with Jarvis off or idle, so
  a game Jarvis slows that far still counts as harm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:12:15 +09:00
javis-bot
dff18c5c7e feat(m0): measure the game's own frame times and try yielding by priority instead of a GPU cap
The 95 % GPU cap is gone: it did not lower utilisation, slowed the LLM and
a YouTube video did not stutter without it. What it was meant to protect
is now measured directly in a game. The new on-demand game_frames stage
records the game's frame times with PresentMon (pinned by hash, game
process only, CSV deleted after reading) with Jarvis off, idle and busy in
the game placement, and busy while Jarvis yields: its processes at
below-normal CPU and GPU scheduling priority, read back to confirm, then
restored. Busy rows run A B B A. The design docs record the candidate and
the pre-registered rule for adopting it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:05:55 +09:00
javis-bot
f66f57f02f fix(m0): keep navigator.webdriver off so YouTube does not stop the video a minute in
With DevTools on (port or pipe), Chrome tells pages a program drives it, and
YouTube then shows "문제가 발생했습니다" after about a minute of video, which
spoilt every window on the gaming PC. Reproduced on Linux: the port alone,
with no command sent, breaks playback at ~40 s; without it, or with
--disable-blink-features=AutomationControlled, it plays on. The browser is now
started with that switch, and one that still reports webdriver is refused
before Jarvis is started.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 03:00:40 +09:00
javis-bot
357f36afcc fix(m0): a player that does not come back spoils its window instead of ending the stage
When loading the page again did not bring a player back, keep_room still
moved the missing player back to the start, the page script threw and the
stage stopped at the first window. Now a failed reload (or a refused
navigation) is recorded as "the video did not play again", only a playing
video is moved back (and the script checks the player exists), so the
window is measured as it is, not counted, and tried again up to the limit
before the stage goes on. The reload waits RELOAD_S for the video.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 02:04:59 +09:00
javis-bot
6d0b3d5bec fix(m0): get a stopped or reloaded YouTube player going again and measure the window again
On the gaming PC the player errored about 38 s into the first window with
Jarvis off; the kit had no way to notice, so every later window measured a
paused, empty player, and after the user reloaded the page the injected
counters were gone and the snapshot script threw, ending the stage early.

- The snapshot reports a lost page (counters gone, or the script could not
  run while the page loaded) and the player's own error screen (first line
  only, bracketed IDs removed) instead of throwing.
- Before each window the kit presses play on a stopped player and loads the
  page again when that is not enough, or when the page was lost or errored.
- A window the player spoilt (error, reload, pause, ad, seek, buffering, no
  frames) is measured again, up to 3 times in all, with the earlier attempts
  and the fixes kept in the row; a lower quality or frame rate is not, since
  a busy GPU can cause it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 01:54:34 +09:00
javis-bot
37e27a2648 fix(m0): count only video windows that played 1080p60 under the cap they claim
- A window counts only when the player kept the asked quality and height
  the whole time and played the source's 60 fps; a player stuck below
  1080p is a stage error, not a note.
- Pauses, seeks and buffering are counted inside the page during the
  window, so a pause that ended before the last snapshot no longer
  passes as GPU stutter.
- A capped window whose cap found nothing to pause, ran no pause cycle,
  or could not pause or resume a process is an error and does not count
  for the cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 01:01:28 +09:00
javis-bot
457b8636fa feat(m0): measure a YouTube video's dropped frames next to Jarvis
User decision (2026-11-11): the 95 % GPU cap is off by default because
gamemode_tune showed it did not lower nvidia-smi's utilisation and cost
6 to 10 % of the LLM's output. The worry behind it was a video stuttering
next to Jarvis, so the new on-demand stage video_drops measures that:

- A public Blender upload plays at 1080p60 in a throwaway Chrome/Edge
  profile (removed afterwards), driven over DevTools with a small
  standard-library WebSocket client, so no package is added to the
  already built venvs.
- Each window counts the video's dropped frames (the player's own
  counter) and the gaps over 50/100 ms between shown frames
  (requestVideoFrameCallback); windows with an ad, a pause, the end or a
  new video are kept but not counted.
- Rows: video alone, Jarvis idle (normal placement with the STT room and
  1 GB for other programs), Jarvis busy (LLM answering, STT transcribing
  ten speakers back to back) without and with the cap in A B B A order,
  then the video alone again. The cap pauses the engine and the STT
  worker only, never the browser.
- run_m0_video.bat runs just this stage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 00:55:26 +09:00
javis-bot
ba662ae199 fix(m0): never leave a process paused when a resume fails under the GPU cap
A resume that failed was only counted, and the process stayed suspended
after the cap ended. DutyCap now keeps what it still owes per process,
does not stack another suspend on it, retries the resume every period
and at the end, treats a raising call as a failure, and reports any
process it could not resume as left_paused (an error for the row).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 23:56:57 +09:00
javis-bot
0e50c558d1 feat(m0): measure the 1 GB room for other programs and a 95 % GPU cap
User decision (2026-11-10, room to watch a video next to Jarvis): the
normal placement leaves 1 GB of VRAM for programs opened later, and
Jarvis's GPU work is capped at 95 %.

- gamemode_tune's normal rows take OTHER_ROOM_MIB off the expert budget.
- gpucap.DutyCap pauses the Strata engine (strata.exe --serve under the
  kit) and the STT worker with their children together for 10 ms of
  every 200 ms (NtSuspendProcess), and always resumes them.
- Every row samples nvidia-smi utilization.gpu; capped rows have an
  uncapped twin (normal with and without the STT, game 2 GB with 2 s
  partials) so the run shows the cost and what the GPU really gets back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 23:52:57 +09:00
javis-bot
3f0ed4343b fix(m0): pace STT partials only in the game placement
User decision (2026-11-10): without a game the STT keeps every partial
pass while the LLM answers. gamemode_tune now measures the normal
placement with the STT room and back-to-back STT only, and paces the
2 GB game placement (back to back as the same-run baseline, every 2 s,
finals only). Docs say the slowdown applies in game mode only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:37:16 +09:00
javis-bot
2a85b6afa5 feat(m0): measure the normal placement with room for the STT and paced STT partials
The gamemode_stt run showed Strata's automatic placement filling the card
next to the STT (15.9 of 16.3 GB, first token 1.6 -> 7.5 s) and the STT's
back-to-back partial passes cutting LLM output to a third. The new
gamemode_tune stage measures both fixes the user chose:

- stt_room: one start without experts (STT loaded) gives the LLM's base
  VRAM; --expert-vram gets what is left after the STT's ten-speaker
  growth (from this PC's stt stage) and a 768 MiB margin.
- stt_pace: a ten-speaker partial pass every 2 s, or none, with finals
  every 0.5 s and finals first; records final wait and STT GPU share.

run_m0_gamemode_tune.bat runs it unattended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:35:23 +09:00
javis-bot
290a4c9dcf fix(m0): record an STT load without timings as a gap, not a pass
A worker that did not stop in time returned its earlier "loaded" note, was
killed, and the placement still counted as ok with no transcription times.
finish() now reports a timeout, and a worker that timed out, died, never
started or measured no ten-speaker or single-clip timing is an error, so
the stage is partial and the next run retries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:16:14 +09:00
javis-bot
3794bb7149 feat(m0): measure the LLM next to the STT, the STT on the CPU and a real game
gamemode now also starts the 2 GB expert placement with a 16K context and
drops the --kv q4_0 and --mtp off rows the Strata 0.1.59 qwen36 engine
ignores; the normal placement answers one request at a time.

gamemode_stt loads the STT worker (stt.py mode=load) before the LLM picks its
placement and keeps it transcribing ten speakers at once while the LLM is
measured, so the VRAM left is the real game share. The game placements run
with the STT on the GPU and on the CPU. gamemode_live repeats the game rows
while the user runs a real game; it only runs when asked for
(run_m0_game_live.bat) and is skipped when nobody is at the PC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:12:53 +09:00
javis-bot
165142be9a fix(m0): measure smaller game placements and read only the latest engine start
The first game PC run showed that a 2 GB expert budget still left a 16 GB
card only 4.3 GB free: the weights, the 32K int8 conversation memory, the
MTP drafter and buffers take about 8.7 GB on their own. The game placements
now also shrink the memory format (q4_0), the context (16K, 8K) and the
drafter, with half the CPU workers (no speed cost measured). The engine log
is appended across starts, so each placement now reads only its own lines;
when no placement leaves the reserve the result names the most free one.
The gamemode runner skips the desktop stage, which is already measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 18:23:28 +09:00
javis-bot
8dfab3ab6e fix(m0,docs): fail gamemode on unstarted placements and never overrun the game reserve
The gamemode stage was "ok" when only the normal placement started; a
placement that does not start, or a measurement error inside one, now marks
the stage partial. The design no longer raises the expert budget above the
computed one: it shrinks the game reserve to what the smallest placement
leaves, shows that, and pauses the LLM during games when nothing is left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 17:43:26 +09:00
javis-bot
13de978cf9 feat(m0): measure the one model's game placements and fix desktop app matching
Add a gamemode stage that restarts Qwen3.6-35B-A3B with a smaller routed
expert VRAM budget (6, 4, 2 GiB, and 2 GiB with half the CPU workers), then
the normal placement again, and records switch time, free VRAM, output speed,
first spoken sentence and 8K-context first token for each. It names the
fastest placement that leaves the game its 8 GB reserve. Strata 0.1.59's
qwen36 engine cannot shrink VRAM while running, so each placement is a restart.

The desktop stage now matches running apps by the process image name Windows
lists (Discord.exe) and closes the calculator it launched by image name when
close-app cannot find a UWP app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 17:39:42 +09:00
javis-bot
c63a859ced fix(m0): fail the agent-desktop speed check when its calls fail
check_mcp timed five CLI and five MCP list-windows calls but ignored their
answers, so a server returning isError/ok=false still scored ok. Read the
envelope of every call, record the error code (never the message), and
fail the check on any failed call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 17:19:33 +09:00
javis-bot
66009d910d fix(m0): read 'no tool for it' as a refusal in the answer judge
The gaming-PC rerun scored the fast model's honest refusals ('...할 수 있는
도구는 없습니다', '...준비되어 있지 않습니다') as failures, so the kit
downloaded two more fast-model candidates for nothing. Rescored with the
fixed judge, UD-Q4_K_XL, UD-Q6_K_XL and Q8_0 all pass 27/27; the
recommended model's Q2_0/IQ3_XXS/IQ3_S still fail 3/27 by calling an
unrelated tool when no tool fits. Record both results in the v2 docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 17:16:12 +09:00
javis-bot
bf4138c9ee feat(m0): measure agent-desktop as the v2 PC-control path
Add a 'desktop' stage to the M0 kit that installs the pinned agent-desktop
1.2.0 Windows build (SHA-256 checked) and checks, on the gaming PC:
calculator buttons by automation id (12+3=15), set-value and read-back on
a test page in a throwaway browser profile, Discord/Explorer read time,
and a warm MCP server against one CLI process per call. Results keep only
counts, timings, error codes and the calculator digits.

Record agent-desktop (UI Automation) as the planned M7 control path in the
v2 design docs, with raw Windows input only for screens without
accessibility.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 17:13:33 +09:00
javis-bot
d2b21a1afa fix(m0): compare whole numbers with sign and decimals in the answer judge
The temperature and minute checks dropped the sign and the decimal part,
so "13.9도" and "-13도" passed as 13 degrees. Numbers are now read whole
("-", "영하" and decimals included) and must equal the tool result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:27:45 +09:00
javis-bot
5da88bdcbb fix(m0): judge the final answer against the tool result, not by substring
The tools stage decides whether to keep a model or measure the next one,
so a made-up answer must never pass. Each case now has a judge for the
spoken answer: numbers must match exactly with their unit (15 or 50
minutes fail a 5-minute timer), weather answers must agree with the rain
in the result and give no other temperature, and a refusal fails when it
also says the thing was done. A new case asks for a timer with no timer
tool offered, the shape Jarvis has today. Unit tests cover the reviewer's
counterexamples and that a lying model makes the stage move on to the
next candidate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:25:29 +09:00
javis-bot
8cb7e57cda fix(reply): never deliver the tool-less answer when the planned call cannot run
When the skipped-tool guard could not run the planned call (not a literal
call, or arguments the tool does not take) it delivered the answer it had
held back, which was given without the tool and could be a made-up value.
The planned call is now recorded as a failed tool result and the model
answers from that, so the user hears that it could not check or do it.

M0 kit: a case passes only when the final answer uses the tool result
(13 degrees, not a made-up 24) or, for the no-tool request, says it
cannot do it; the no-tool request now counts. The kit mirrors the
engine's failed-result path, and a failing model moves on to the next
candidate on these stricter checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:19:30 +09:00
javis-bot
85a562c171 feat(m0): try the next model when tool calling still fails with the fix
User rule: if the tool-calling fix does not work, change the model. The
tools_fast / tools_recommended stages now measure the next, less
compressed model of the same family (fast: UD-Q6_K_XL then Q8_0;
recommended: IQ3_XXS then IQ3_S) when the default still gets a scored
case wrong, stop at the first one that passes every case, and record it
as tools_verdict.chosen. Candidates without enough free disk are skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:09:58 +09:00
javis-bot
903e0eb9de fix(reply): stop models answering past an available tool
M0 on Strata caught both v2 models skipping a tool they had been given:
the recommended model made up "24 degrees, sunny" instead of calling the
weather tool, and the fast model said "timer set" without calling the
timer tool (3 of 3 each). qwen2.5:7b does the same through the engine.

- Prompt: TOOL_HONESTY_RULES in the persona (live facts only from a tool
  result this turn, actions only claimed after a tool reports success, no
  answer text next to a tool call), restated at the end of the system
  message by tool_honesty_reminder() with the turn's real tool names.
- Engine: skipped-tool guard. When a native-tool model answers before any
  tool has run and the plan names a callable tool as a concrete call, the
  answer is dropped and the plan steps are direct-executed, then the model
  answers from the results. Only literal, schema-valid steps run (regex
  fast path, no LLM repair); otherwise the held-back answer is delivered.
- Planner: the AVAILABLE TOOLS lines now carry argument names
  (planner_tool_catalog), so concrete steps use the tool's real keys.
- Eval evals/test_tool_skip_confabulation.py; on qwen2.5:7b, 3 runs each,
  available-tool cases went from 0/12 to 9/12 calling the tool.
- M0 kit: tools_fast / tools_recommended stages (run_m0_tools.bat) rerun
  tool calling before and after the fix on the real models, loading the
  production system_prompt.py and planner.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:05:40 +09:00
javis-bot
e0b172e796 feat(m0): check a virtual monitor for the bot browser window
The user wants the bot window on no real screen at all. Add a vdisplay
stage that drives the Parsec Virtual Display Adapter already on the PC
(no install, the display dies with the script), moves the clock window
onto it as a normal and as a tool window, and asks about real monitors,
Alt+Tab, Discord's Screens tab and whether the cursor can wander there.

Record the M0 defaults the user confirmed in docs/v2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 13:29:58 +09:00
javis-bot
dc2e709d31 fix(m0): keep Strata's engine log when an LLM start fails
The first gaming-PC run failed the recommended model with only Strata's
one-line reason ("no CUDA-capable device"), which is the picture
reader's side message, not the cause. The engine log stayed on the PC.

A launch that does not become ready now copies the newest
*.engine.log (scrubbed, last 200 KB) into logs/, which the results zip
already packs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 07:04:38 +09:00
javis-bot
ff9b868e62 fix(m0): bound context-test prompt size when tokens are under-reported
The context test re-estimated characters per token from the server's
prompt_tokens after every request. A server that reports a tiny count
made each next prompt grow by orders of magnitude, so the kit tests
reached 8 GB and were killed by the host memory guard. Clamp the
estimate to 0.5-4 characters per token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 17:20:01 +09:00
javis-bot
5762a22a67 fix(m0): retry failed LLM starts and never measure a foreign server
- A Strata install or start that never reached a server is saved as
  "failed" (not "partial"), so the next run retries the stage.
- A launch that exits non-zero is not ready, even if something answers
  on the port afterwards.
- Before the cold start, the stage stops if another program already
  listens on port 8080 and tells the user to close it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 15:53:40 +09:00
javis-bot
653d6ecc03 feat(m0): add gaming-PC measurement kit for v2 defaults
One double-click kit (scripts/m0/run_m0.bat) that measures the gaming PC
and packs the results into a single zip:

- PC info (GPU, driver, RAM, disks, monitors)
- Windows checks with the user: bot browser window hidden from Alt+Tab
  but listed in Discord's share picker and still drawing while covered;
  Discord pop-out stream window identification and occlusion; installed
  games, anti-cheat services and the borderless game detector
- MeloTTS, CosyVoice2 and Fun-CosyVoice3 Korean synthesis speed
- faster-whisper large-v3-turbo: final/partial latency, CER, wake-word
  hint, batched decoding for several speakers
- Strata LLM (fast and recommended presets): TTFT by context length,
  decode speed, tool calls, picture reading per quality tier, Korean
  answers, parallel requests, VRAM give-back and restart/switch timings

uv and Python, per-stage venvs, models and Strata all live inside the
work folder. Results are scrubbed of user/computer names and home paths.
Verified on Linux GPU (LLM client against Ollama, STT, MeloTTS,
CosyVoice2/3) and on the Windows 11 CI VM (window checks, game scan,
MeloTTS/CosyVoice2/STT installs, Strata install and plan).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 15:38:41 +09:00
javis-bot
ffc16665e5 fix(controlBrowser): never report moveMouse/search success without a real move
Some checks failed
Release / semantic-release (push) Successful in 19s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m15s
Release / build-linux (push) Failing after 7m7s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
moveMouse returned ok:true even when humanHover did nothing (no on-screen box)
or the selector never matched — recreating the "claims it moved but didn't"
bug. Now: humanHover returns a boolean (and brings the element into view first);
moveMouse returns ok:false when the target isn't found or has no on-screen box,
and when site=naver/... whose box isn't on the current page it navigates to the
site home first before hovering. search now reports input=human|api-fallback|api
so a silent fallback to cursor-less DOM input is visible, and the tool surfaces
that note in the reply instead of implying a human-like search happened.
2026-06-24 19:17:46 +09:00
javis-bot
5629da7e9f feat(controlBrowser): add moveMouse (hover) action for the visible cursor
The tool had no cursor-move/hover action, so "move the mouse to the search box"
had nothing to call and a weak model just claimed it had moved it. Add a
moveMouse action wired to human.humanHover (real xdotool cursor), targetable by
CSS selector or site= (that site's search box). Also clarify in the tool
description that 'navigate' types into the address bar (no mouse) while
'search'/'type' move the real cursor to the on-page box and click before
typing, so the model picks the visible-cursor path when the user wants it.
2026-06-24 19:14:04 +09:00
javis-bot
83999a5b0b fix(prompts): classify every sub-8B model (2b/4b/5b/6b) as SMALL
Some checks failed
Release / semantic-release (push) Successful in 24s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m21s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
detect_model_size only matched 1b/3b/7b, so a genuinely small model such as
qwen*:4b fell through to LARGE and got the terse, less-guided prompt set it
can't follow — contributing to off-tone, rambling replies. Extend the small
patterns to cover all sub-8B sizes (1b-7b) across :/-/_ separators and sync the
spec table.
2026-06-24 18:31:54 +09:00
javis-bot
d5fd218c86 fix(reply): stop weak models parroting persona example facts
A 4b model replied to "하이" with "테니스 연습을 Trenches Gym에서..." — it copied
the literal "box at Trenches Gym" few-shot example embedded in the persona
prompt and mangled boxing into tennis, presenting a prompt example as if it
were a real user fact. Remove the copyable proper-noun example and add an
explicit guard: use ONLY names/places/activities that literally appear in the
memory section, never borrow them from the instructions or example wording.
2026-06-24 18:31:54 +09:00
javis-bot
7bb9718c34 feat(reply): cap spoken replies at a single sentence
Replies stayed long because the prompt stack gave conflicting length signals:
the persona said "one sentence (two at the very most)" AND told the model to
"state the answer in a sentence, then add a dry observation" (a 2nd sentence),
while voice_style said "two to three sentences maximum". The model followed the
longest. Make all three sources agree on exactly one sentence: the persona's
aside must now fold into the same sentence as a trailing clause (never a 2nd
sentence), voice_style caps at one sentence, and agents/llm.md says 한 문장.
Shorter replies also cut Edge-TTS latency, since synth time scales with text
length. Specs (prompts.spec.md) and docs/llm_contexts.md updated; deterministic
prompt-contract tests added.
2026-06-24 17:55:27 +09:00
javis-bot
b52ffd2b18 perf: run auxiliary LLM calls on a small model, big model only for the answer
Some checks failed
Release / semantic-release (push) Successful in 19s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / build-linux (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
tests / Unit tests (Linux, Python 3.11) (push) Has been cancelled
Intent judging, tool routing and arg extraction are classification/JSON calls,
not the spoken answer, yet the stack aliased OLLAMA_INTENT_MODEL back to the big
chat model — so each command paid the big model's cost 2-3 times for routing
before the reply even ran. With the GPU on, that round-trip stacking is the main
remaining per-turn latency. Default OLLAMA_INTENT_MODEL to qwen2.5:3b (the
project's reference small model, clean Korean on classification) and have
ollama-init pull it. The reply engine already routes these calls through
intent_judge_model, so answer quality is untouched; set OLLAMA_INTENT_MODEL =
OLLAMA_CHAT_MODEL to fold back onto one resident model.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 17:35:40 +09:00
javis-bot
140fc56f18 feat: play the Nth YouTube result in browseAndPlay via an index arg
Some checks failed
Release / semantic-release (push) Successful in 22s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m10s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
agents/llm.md promises "play the Nth video from the top", but browseAndPlay
only ever clicked the first result. Add an optional 1-based index argument
(default 1, backward-compatible) threaded to the Node helper, which now clicks
the Nth a#video-title and clamps to the number of results returned so asking
beyond the list plays the last available video instead of failing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 15:33:45 +09:00