Commit Graph

103 Commits

Author SHA1 Message Date
javis-bot
1400a3ff97 fix(app): the LLM placement measures only what it can trust and keeps a game the 0 GiB step
- a measurement whose short answer fails, or whose second card reading fails or shows nothing
  more held, is not kept: the model starts with expert-vram=0 and the next start measures again
- measurements are kept per context and requests at once (<context>/<parallel>) and per card
  (its NVIDIA UUID), not by the card's size
- a server on this app's port left from a run that ended badly is stopped before the card is
  read, so its memory is not counted as other programs'
- the game placement falls back to 0 GiB of experts before pausing when 2 GiB leaves a game
  nothing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:01:53 +09:00
javis-bot
af6f2c855d feat(app): place the LLM's experts so the speech-to-text worker keeps its room
- with an NVIDIA card the app sets --expert-vram itself by the M0 kit's formula: what the card
  holds before the start, the model without experts, the STT worker's share (2740 MiB), the
  engine's margin (768 MiB) and room for programs opened later (game.otherRoomGb, default 1 GB)
  are left free; Strata's own placement filled the card and left the STT no room
- the model without experts is measured once per context and card (a start with
  expert-vram=0, one short answer, the card's memory and the engine log's staging), kept in
  config/llm-probe.json; the real start replaces that server
- placement.rs also works out the game placement (the reserve, else the 2 GiB minimum and the
  room a game really gets, else pause) for the switching that comes next
- with the game PC's M0 numbers the normal placement comes out at 1.5 GiB, as M0 measured

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 09:53:20 +09:00
javis-bot
7e83b09dd5 fix(app): game detection finds Battle.net games where they are installed and takes any letters in a path
- Battle.net installs anywhere: its games come from the machine's uninstall entries (removed by
  its Blizzard Uninstaller or published by Blizzard Entertainment, with an install folder, not
  Battle.net itself), read when the watch starts; a program inside one is a game named after
  the entry
- launcher folders are matched part by part with ASCII case ignored; the name was cut by byte
  positions of a lower-cased copy, which panicked the detection thread on a path such as
  D:\İ\steamapps\common\하데스\game.exe
- checked on the win-ci VM: a Steam game under a non-ASCII path and a Battle.net game (test
  uninstall entry) turn game mode on with their names and off when they end

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 09:32:53 +09:00
javis-bot
c9101e0d6a feat(app): game detection for the performance stages
- while the services run the app looks at the foreground window every 2 s (Windows): the
  person's notGames and games lists first, never a browser, video player, the desktop, Discord,
  a launcher's own window or this app; a program under a Steam, Epic, Riot or Battle.net game
  folder, exclusive full screen, or a window covering a monitor without title bar or border whose
  process uses at least 10% of the GPU's 3D engine (the M0 rule) is a game
- game mode stays until that game's process ends, also behind other windows; the dashboard shows
  "게임 모드: 켜짐 (name, reason)" and the game log tab its changes
- a 게임 모드 settings screen (detection on/off, the two lists)
- checked on the win-ci VM: a program in a Steam game folder turned it on after 1.4 s and off
  2.0 s after it ended

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 09:08:50 +09:00
javis-bot
1bd7defbb4 feat(app): the bot browser's window on a virtual monitor to share in Discord
- features.browserShare (봇용 크롬창 화면공유 선택 표시, off by default) and a driver card on the
  features screen: the Virtual Display Driver 25.7.23 (MIT, SignPath-signed) ships in
  display-driver/ (hash-checked by package.sh); install and uninstall run jarvis.exe elevated
  once (--display-driver), settings in %ProgramData%\JarvisBot\display-driver, another
  program's copy of the driver is left alone
- the monitor is on the desktop only while the bot browser uses it, at the rightmost real
  monitor's bottom-right corner; the stop closes Chrome first, then takes the monitor off
- Chrome starts with no window and the app opens it over CDP in the background, so it never
  takes the focus; the window is a tool window (no Alt+Tab, no taskbar button), fills the
  monitor's work area and keeps drawing when covered; without a monitor it runs headless with
  the reason on the dashboard
- Chrome runs with --do-not-de-elevate on Windows: started by an app running as administrator
  it restarted itself as the user and the app lost hold of it
- checked on the win-ci Windows VM from install to uninstall (focus kept while typing, no
  taskbar button, no Chrome left, nothing left after uninstall) and on Linux with a real Chrome

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 02:43:32 +09:00
javis-bot
56212c06dc fix(brain): botBrowser keeps a tab per person, one deadline, and never repeats an action
- each asker has a tab and connection of their own (at most 5, least recently used closes first)
- links, forms and window.open that would open a new tab load in the same tab
- one 15 s deadline covers connecting, the action and the page; events do not extend it
- a broken connection retries only open and read; click, type and back may have run and are not repeated
- <<< and >>> in page content are replaced so it cannot close the fence
- the description says how to search on a site (open, then type with submit=true); eval case 0/3 -> 3/3 on qwen3:8b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 22:33:00 +09:00
javis-bot
077db5031c feat(brain): botBrowser tool drives the bot's own Chrome over CDP
open, read, click (label or [n]), type (optionally Enter) and back; page content comes back fenced
as untrusted data; only http(s) addresses; no LLM call. Offered only when the app started the bot
browser (JARVIS_BROWSER_CDP) and the browser feature is on; meanwhile fetchWebPage, which has no
address guard, is not offered. Eval evals/test_bot_browser_tool.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 22:18:05 +09:00
javis-bot
24d0798bae feat(app): the bot's own Chrome in the data folder, reaching only public addresses
- install step browser: Chrome for Testing 157 (pinned SHA-256) into runtime/chrome
- services start it headless with its own profile when features.browser is on; CDP on a free
  loopback port passed to the worker as JARVIS_BROWSER_CDP; a failure leaves the services running
- every Chrome connection goes through the app's loopback egress proxy: one lookup, public
  addresses only (403 otherwise, no host logged), QUIC off, WebRTC only through the proxy
- stop: Browser.close, then kill after 5 s; dashboard shows the browser's state
- the worker's runtime environment follows the settings step, not every step being installed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 22:01:04 +09:00
javis-bot
9185265010 fix(app): collection connects only to the addresses it checked, within 15 s, logging no address
- the fetcher resolves once, requires every address to be public and connects only to those
  (TLS still checks the name), so DNS rebinding cannot reach the local network; redirects too
- 15 s for the whole fetch also against servers that send slowly; no URL in any log line
- a site whose title alone changed is kept again
- target names that differ only in case are refused (one Windows folder)
- the worker always gets the collection file, so results older than the keep days go also
  with no targets or the feature off
- the results screen lists every target with results, not only those in the newest 100

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:39:04 +09:00
javis-bot
b94e9dc112 feat(app): background collection frame for sites, search words and RSS feeds
Targets (none by default), interval and keep days on a new 백그라운드 수집 screen with the
results; the worker collects on its own thread through the web search's address check and
writes only to the data folder's collect/ (logs carry target names and counts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:29:49 +09:00
javis-bot
3ef4281180 feat(app): features screen with tool switches and MCP servers; v2 drops host file and screen tools
- Settings `features` (weather, webSearch, meals) and `mcpServers` (name, command, args, env names,
  enabled), edited on the new 기능 screen. The tools of a feature that is off go into the brain's
  `disabled_tools`; only enabled MCP servers go into its `mcps`.
- MCP environment values (API keys) live only in the encrypted secrets (`mcp:<server>:<NAME>`),
  shown as stored or not, masked in logs, removed with their server, and reach the worker only in
  `JARVIS_MCP_ENV`, which the brain takes out of its environment before giving each server its own.
- `enabled_builtin_tools(cfg)` is the one list the router, toolSearchTool and the executor use, so
  a turned-off tool is neither offered nor run, also when the model names it.
- The v2 brain never offers v1's `localFiles` (home folder read/write/delete) and `screenshot`:
  anyone in the voice or chat channels could ask for them. Screen viewing comes in M7.
- The brain connects MCP servers on its own thread and logs each outcome without debug logging.
- Stored secrets are masked in the log hub as soon as the data folder is found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 09:56:17 +09:00
javis-bot
0d9c6ec984 docs(v2): M5 recheck on the game PC with the UTF-8 option off
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:44:26 +09:00
javis-bot
91d0109139 feat(app): chat channel name/topic sync, TTS speed, safe window close
Review follow-ups to M5.

- Chat channel names and topics (app/src/channel_sync.rs): a channel with
  syncDiscord (default on) gets its name and description as its Discord
  name and topic when saved; registering fills them from Discord;
  Discord's form of the name is written back; a topic over 1,024
  characters is refused before saving; a rate-limited edit is pending and
  retried once by itself with the latest text; a missing permission fails
  with the reason and the app keeps the text. Commands
  discord_channel_info and channel_sync_status. Checked on real Discord
  with a temporary channel (fill, edit, rate limit, automatic retry).
- voice.tts.speed (0.5-2.0) on the model screen, passed to the worker as
  --melo-speed. MeloTTS's Korean model has one voice, so no voice choice.
- Closing the window runs AdminApi::shutdown first: an install in
  progress is cancelled and waited for, the services stop, a failed run's
  leftovers (Strata) are cleared (Controller::shutdown). If that fails the
  window stays open with the reason; closing again quits.
- WebView2 data goes in the data folder (cache/webview), or a temp scratch
  folder before one is chosen, removed at the next start: %APPDATA%\JarvisBot
  holds only the location file.
- Command output that is not UTF-8 (CP949 on a PC without the UTF-8
  option) is decoded in the OEM code page instead of ending the reading.

Verified on the win-ci VM: closing during the install leaves no process,
no --location keeps only location.json under %APPDATA%, the unit tests on
Windows 168/168.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:23:26 +09:00
javis-bot
76144fb19b fix(app): Windows stop check, size-based download progress, APPDATA in the data folder
Found running the app's admin commands on the game PC (Windows, RTX 5070
Ti): the full install, Strata started by the app downloading the 22.9 GB
model, the worker and Discord up, five spoken questions all answered.

- Windows refuses a connection to a closed local port only after about
  2 s, so the 2 s request timeout made every stop "not confirmed". The
  check now waits up to 10 s to connect and still counts only a refusal
  (not a timeout) as stopped.
- Strata 0.1.59 reports the model download as "X of Y GB"; percent()
  reads that, and the dashboard shows the current line's progress only
  (it had stayed on the engine download's 80 %).
- Strata and the NVIDIA shader cache write under %APPDATA% whatever
  STRATA_* says: the layout sets APPDATA inside the data folder on
  Windows, next to LOCALAPPDATA.

Roadmap: M5 step 5 done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 18:51:12 +09:00
javis-bot
5c510bb602 feat(app): start Strata from the data folder and point the brain at it
M5 step 4, second part. The services start the LLM first: the installed
Strata with the M0 kit's model and normal placement (models in
models/llm, parallel 1, or 2 when a chat channel answers every message,
pictures off, --restart), wait until /v1/models lists the model, and stop
it last. The first start downloads the model; the dashboard shows
Strata's last line and percentage, and stop also cancels a start (the
server is cleared). llm.url uses a server the app does not start.

On every start the app writes config/brain.json (Strata's /v1, the
listed model id, memory/jarvis.db) and hands it to the worker as
JARVIS_CONFIG_PATH. Loading a config with its own db_path no longer
creates ~/.local/share/jarvis.

service_start/service_restart run in the background. Settings gain
llm.context and llm.port (model screen) and llm.url (file only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:45:38 +09:00
javis-bot
a4436698c8 feat(app): M5 step 4a, install the runtime and voice models into the data folder
- download: resumed downloads checked against a pinned SHA-256.
- install: uv 0.11.20, the worker/brain venv (Python 3.12, the new
  src/voice_worker/requirements-brain.txt), the MeloTTS venv (Python 3.11,
  PyTorch cu128), Whisper large-v3-turbo, the Korean MeloTTS voice and
  Strata, the M0 kit's recipe; markers per step and version, cancel, retry,
  everything (caches, temp, HF and NLTK data) inside the data folder, the uv
  cache cleaned and the worker settings pointed at the venvs when done.
- The services give the worker the same environment once installed.
- The wizard's model step shows and drives the install.

Installed from an empty folder on Linux and started the services on it:
worker connected, STT and TTS ready, nothing written outside the folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:11:23 +09:00
javis-bot
20300c017e feat(app): M5 step 3, the management app's screens and their commands
- admin_api: the one entry point of the screens (app status, data folder,
  PC check, Discord token check and pickers, settings, service control and
  the dashboard, logs query, error report and zip export).
- discord_setup: token, Message Content and invite link checks over REST.
- app/ui: first-run wizard, dashboard, five settings screens and the log
  screen in plain HTML/CSS/JS (dark theme after DESIGN.md, Korean text).
- jarvis-admin-dev: a loopback, token-guarded development server so the
  screens run in a browser against the real commands.

Checked in a browser with the real test bot and voice worker: wizard
steps, start/restart/stop, settings save, chat channels, logs and export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 14:15:58 +09:00
javis-bot
0973ad9815 feat(app): M5 step 2, service control and the dashboard data
- control: one Controller starts, stops and restarts the services (stopped,
  starting, running, stopping, failed); a failed start leaves nothing
  running and keeps its error with secrets masked. The app binary uses it.
- dashboard: built from the core's events alone: the bot's tag, the voice
  channel, live captions (screen only, cleared on leaving or stopping), the
  last 20 voice answers split into stages with medians (matched by the new
  AnswerAsked event), and missed requests without text.
- system: memory in use and per NVIDIA GPU its use and VRAM (nvidia-smi).
- DiscordCore::voice_members, WorkerClient::hello, the conversation's missed
  requests shared with the dashboard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:49:51 +09:00
javis-bot
390f6f0b86 feat(app): M5 step 1, the admin backend under the management app
- datadir: the data folder layout, the location file (JARVIS_DATA_DIR wins),
  free space for the wizard.
- settings_edit: the fields the five settings screens edit; apply writes only
  non-defaults, keeps unknown keys and writes nothing unless the app's own
  check passes. A memory section reaches the worker's hello.
- secrets: config/secrets.dat, DPAPI for the Windows user; the services take
  the bot token from it before JARVIS_DISCORD_TOKEN.
- logs: the log hub behind the log screen (tabs, daily files kept 14 days,
  the recent error report, a zip under 10 MB, secrets and token-shaped text
  masked); the log macros and the worker's output feed it.

Spec: app/src/admin.spec.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:36:02 +09:00
javis-bot
07c701be54 fix(reply): a tool call written in Qwen's XML keeps its argument types
Every <parameter> value was passed as text, so a delete confirmed with
confirm=true reached chatChannels as "true" and asked again. Values now take
the type the tool's input schema declares (boolean, integer, number, array,
object); a value that does not fit stays text. The roadmap's eval summary now
names the reply-length failure apart from the later passing runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:21:45 +09:00
javis-bot
6d435caef0 docs(v2): M4 latency work and the second live measurement
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:16:41 +09:00
javis-bot
e6ee152774 fix(voice_worker): plan chat tool turns, and keep v2 answers to the core
- A chatChannels pick from the router counts as a tool turn for the intent
  judge ("post this in the schedule channel" needs the planner).
- The default-language directive names an explicit language request as its
  one exception; the history block says plans and appointments people
  mentioned need no calendar tool; why/how answers give only the core reason.

Evals (Qwen3.6, v2 options): reply length 3/3 (v2 185-205 chars, v1 315-356),
language, history, chat channels and tool-skip 22/22; full suite 219 passed,
6 failed (paris/london, graph branch and four v1 single-user memory cases).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:06:40 +09:00
javis-bot
46e4e75d05 feat(voice_worker): a short voice prompt and one judgement before each v2 answer
- compact_prompt: the v2 persona in a tenth of v1's words, answers as short as
  possible, the full tool guidance only on turns with a data tool. A greeting's
  reply prompt went from about 4,500 to 1,400 tokens on Strata.
- intent_judge: the tool router judges whether the turn needs tools; the
  planner runs only when it does. prepare_intent runs the pick ahead on a
  partial transcript (worker message prepare), and the reply reuses it.
- Without stream_browser the screen-share tools are not offered.
- The time context carries the 12-hour form (10:37 was read as pm).
- A <tool_call> block written as the reply is parsed as a call, not spoken.

Evals (Qwen3.6 on Strata, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1):
221 passed, 4 failed (known failures); new evals/test_voice_reply_length.py
(v2 answers 234 chars against v1's 416) and test_reply_language_default.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 12:25:15 +09:00
javis-bot
7623d6d002 feat(voice_worker): the v2 brain answers in Korean unless asked for another language
The brain passes default_language (worker --reply-language, default Korean) to
the reply engine; unlike the output_language lock it gives way when the user
explicitly asks for a language. Tools get the ko code (--tool-language).

Eval evals/test_reply_language_default.py (Qwen3.6 on Strata): 6/6, English
questions answered in Korean, an explicit English request in English, and the
control without the default answers English in English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:34:12 +09:00
javis-bot
5507ec9b71 test(app): time the whole conversation on real Discord, alone and with 9 more voices
live_round_trip.rs runs the app's services with the real worker and brain,
plays recorded questions from a second bot and times the final, the
answer text, playback start and the first audible frame after the speech
ends; a second worker session streams nine more voices for the load run.
Roadmap: first M4 step 4 numbers (median 3.5 s alone, 4.4 s under load).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:58:55 +09:00
javis-bot
bd2eb49e3d fix(app): a wake word said alone waits for what the speaker says next
A pause after the name ("자비스, ... 지금 몇 시야?") is longer than the
worker's 300 ms end silence, so the wake word ended its own utterance:
Jarvis answered the bare name and ignored the question that followed (no
wake word). Found on real Discord. A voice utterance that is only a wake
word is now joined to the same speaker's next final when that utterance
is running or starts within call.followUpMs (default 1500); otherwise the
wake word alone is the request after that time. Stops drop it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:44:06 +09:00
javis-bot
9ff307d7fe fix(voice_worker): the v2 brain answers in the asker's language
v1's tts_engine default (English-only Piper) made the reply engine force
English, so Korean questions got English answers. The v2 brain speaks
through the worker's MeloTTS and chat answers are not spoken, so it runs
with tts_engine=melo; an output_language lock still wins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:16:34 +09:00
javis-bot
19f984ae73 fix(voice_worker): one person's voice and chat answers are written one at a time
The voice and chat lanes let a person's voice reply start while their chat
reply was still running. The chat delete consent counts "the very next
message" by turn number, so a message that started before the delete was
asked could pass as the confirmation, and the person's dialogue memory
was updated out of order.

- A reply takes its asker's lock, then its lane's: one person's replies
  stay one at a time in the order asked, different people's voice and
  chat answers still overlap, and a reply waiting for its asker holds no
  lane. Test: a person's voice question starts only after their chat
  answer.
- Specs and roadmap: the lanes are per person; the roadmap no longer says
  voice answers wait behind chat answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:27:13 +09:00
javis-bot
a70ccda37e feat(app): answer registered chat channels in chat
- chatChannels[].answer: wakeWord (default) or everything. Chat uses
  call.chatWakeWords (else the voice wake words) with the shared
  designated-people switches.
- The brain's answer (reply via chat, not spoken) is posted as a Discord
  reply to the message, pinging nobody; failed or empty answers post
  nothing. Bots, channels with sending off and stops are not answered.
- Chat runs beside the voice conversation and never takes its turn.
- Core: send_chat_reply, chat_answer and the ReplyText event; the worker
  client's reply takes via.
- Specs (conversation, discord core) and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:36:43 +09:00
javis-bot
f892a1dce9 feat: answer calls in the voice channel through the response policy
- conversation.rs: the loop between the core's events and the rest.
  Finals go through call detection and the response policy; answers are
  asked of the brain and spoken from the app's queue, an interruption
  cancels only that answer, a voice stop silences everything, and missed
  requests are kept without text (last 100). Tested with a fake voice side.
- Core: answer(user, text) adopts a playback request the worker speaks
  under the same id, cancel(id) stops one request, ask_addressed and
  ask_rank pass the conversation's questions; new SpeakingStopped,
  Addressed and Ranked events.
- Worker client: reply, addressed and rank messages and their results; a
  reply with no answer, refused or not sent ends its speech as failed.
- Voice worker: addressed and rank messages on their own threads, beside
  a running reply; a spoken reply cancelled while the brain is answering
  is not spoken. judges.py holds the two short LLM judgements with the
  heard text fenced as data; eval evals/test_voice_judges.py.
- Specs (conversation, discord core, voice worker), docs/llm_contexts.md
  and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:26:47 +09:00
javis-bot
6fc3983a40 feat(app): call detection and the four response policies
- calls.rs: wake words anywhere in the utterance with near matches by
  edit distance on NFKD-normalised letters (Hangul splits into its
  letters), call modes (wakeWord, addressed, everything), designated
  people with both switches, and a voice stop (wake word + stop word).
- policy.rs: first, inOrder, latest and important as an event-driven
  state machine with the time passed in: ordering by end time, waiting
  for an earlier-ended utterance still being transcribed, queue limit,
  interruption, importance ranking only on a tie with a timeout, stop
  dropping the queue, and a reason for every missed request.
- settings: new call and response sections, validated with the key.
- conversation.spec.md documents the contract; registry and roadmap
  updated. Wiring into the Discord core and the brain comes next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:05:50 +09:00
javis-bot
b88b28140f docs(roadmap): match the M3 eval failure notes to the run logs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:57:54 +09:00
javis-bot
3ec430fd25 fix(reply): known user location reaches getWeather; retries are not blocked by the previous reply
- Drop the engine's deterministic Korean weather shortcut. It bypassed the
  chat model and the tool runner, so a location the user had stated could
  never be used. The tool already returns one clean Korean sentence.
- getWeather and the tool-argument prompts pass the place the user has
  said they are (conversation or memory) and leave the argument empty
  otherwise, so the detected location is not copied into it.
- getWeather retries a "district, city, country" name with its leading
  parts dropped when the geocoder finds nothing.
- The planner gets the live time/location line and plans
  searchMemory topic='where the user lives' only when the location is
  unknown or disabled.
- The per-reply repeat guard counts only this reply's tool messages, so
  calls carried over from the previous reply no longer block a retry the
  user asks for.
- Memory extractor: the context block sits after the examples and the
  no-location example uses a different query, so a location already in
  context is not asked for again. New counterpart eval checks that it is
  still asked for when the context has none.
- Tool retry eval fails every call in turn 1, so a model that retries by
  itself inside turn 1 still exercises the user's "try again".
- Specs (planner, reply), docs/llm_contexts.md and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:55:45 +09:00
javis-bot
4bb3895ca0 docs(roadmap): record the eval run on Strata (Qwen3.6-35B-A3B) from the game PC
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:21:23 +09:00
javis-bot
a45e198df6 docs(roadmap): record the M3 test and eval check
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 17:19:08 +09:00
javis-bot
4621e6c1fd fix(chat): the delete consent check knows the pending message and reads the whole reply
The consent classifier is now told which message and channel are pending, so a yes for another
message ("yes, delete message 22" while 21 is pending) is a no and cancels the request. A reply
longer than 600 characters is no longer cut and judged by its start (which could drop a later
"don't"); it counts as no without a model call.

Eval evals/test_chat_delete_consent.py gains the different-message and long-reply cases:
17/17 on qwen2.5:7b, three runs. The previous prompt said YES 3/3 to
"응, 일정 채널에 있는 거 지워" while a message in 잡담 was pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:26:58 +09:00
javis-bot
0182823592 fix(chat): delete needs the person's own yes; chat action ids never repeat
- chatChannels delete with confirm=true now runs only in the person's very
  next turn and only when their own message of that turn agrees, checked by a
  small fail-closed YES/NO call on the router model chain (the message fenced
  as data). A refusal, question, other topic or error cancels the request;
  an unconfirmed request lapses after one turn. The reply model's confirm
  flag alone can no longer delete.
- chat_action ids carry a per-run nonce and the session, so a late answer
  meant for an earlier connection (or worker run) never completes a newer
  action.
- Tests: agree, refuse, other topic, lapse, classifier failure, late answer
  across reconnects. Eval test_chat_delete_consent: 12 of 12 on qwen2.5:7b,
  3 runs. Spec and llm_contexts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:08:35 +09:00
javis-bot
fe4e296361 feat(chat): chatChannels tool for the registered Discord chat channels
- Tool (chat_channels.spec.md): list, read, search, send, edit, react,
  delete, pin, unpin. Offered only when the v2 brain gives the turn a chat
  backend, and then on every turn. Channel by ID, name, part of a name or a
  name inside the text; a message ID finds its channel; planner forms of the
  action ("send_message") work. Read and search results are fenced as
  untrusted data. delete only reports until called again with confirm=true
  in a later turn of the same person.
- Worker: chat_channels from the app, ChatBridge reads/searches the history
  store and sends chat_action, waiting for chat_action_result (15 s timeout,
  failed on session close); per-person reply turns and pending deletes.
- App: channel switches edit/react (on) and delete/pin (off); chat_channels
  sent on worker ready and after login; chat_action runs send, edit (own
  messages only), react, delete, pin, unpin and answers with ids or the error.
- Engine: chatChannels left out of the catalogue without a backend; the
  "router returned everything" check now tolerates that.
- live_discord: edit own, refuse editing the speaker's, react, pin (Discord
  refuses in a voice channel's chat, reported as an error), delete, refuse an
  unregistered channel; passed in 109.4 s.
- Eval (qwen2.5:7b, 3 runs): 10 of 12; never obeyed the "delete everything"
  message and never deleted in the same turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:01:33 +09:00
javis-bot
b3dcd12ec9 fix(voice_worker): keep history and the brain's output scoped to who was there
- A final is stored under the channel, speaker name and time fixed when its
  end was decided. Decoding takes a while; a move, rename or late joiner
  meanwhile used to change where and when it was said, and who heard it.
- At start the worker closes presence a killed run left open
  (HistoryStore.close_open_presence), so a person who is gone no longer
  hears new speech after a restart.
- run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm
  profile, extractor, diary, graph, digest); the v2 brain uses it so another
  user's profile or diary never reaches the prompt.
- While a brain reply runs, stdout and stderr are muted (also helper threads
  such as the web fetch pool); the worker logs through voice_worker/log.py to
  the original stderr.
- Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the
  prompt in earlier runs; with an 8k tag the chat-record question was answered
  4 of 4 and nothing reached the output. Noted in llm_contexts and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:28:25 +09:00
javis-bot
864b2e7fdb feat(voice_worker): record history from the app and answer reply requests with the brain
Worker (voice_worker.spec.md "History and brain"):
- --history <file> keeps the conversation history store; --brain hosts the
  jarvis reply engine (JarvisBrain, settings point ollama_base_url at /v1).
- voice_join / voice_leave / member_join / member_leave keep presence; the
  session end closes it. Non-empty finals are stored under the speaker's name
  in the current voice channel.
- chat_message / chat_edit / chat_delete keep registered chat channels.
- reply builds the asker's scoped history context, runs the brain on a thread
  (one at a time), answers reply_text, stores a voice answer, and with
  speak=true says it under the same id. memory settings in hello and
  memory_config. brain_unavailable without --brain.

App (discord_core.spec.md "History reporting"):
- WorkerClient sends the new messages; the core reports the voice channel
  and its heard members after a join, a move and every worker reconnect,
  member joins and leaves, voice_leave when a session ends, and every message
  of a readable registered channel (its own too, with `to` on replies), edits
  and deletes.
- live_discord runs the worker with --history and checks the store at the
  end: 8 finals under the speaker's name, the edited chat message only,
  Jarvis's own messages, 5 presence intervals, none left open (passed, 107.8 s).

Eval: history recall gains a chat-record case and fails a reply that reads a
record stamp out (field run with qwen2.5:7b did once in four).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:16:24 +09:00
javis-bot
7a288129de fix(memory): rank the whole history and keep relevant recent records
- Search scored only the newest 5,000 matching records, so an old record
  behind enough newer chatter was never found. Both passes (word rarity counts
  and the score) now run in SQLite over every record in scope and only the
  best rows come back.
- context() excluded the recent window from search and then cut that window
  for the budget, so a relevant recent record could vanish. Search now covers
  the recent window, matches keep their half of the budget, and a matched
  record is removed from the recent block so it appears once.
- Matching used SQLite's ASCII-only lower() against Python's lowercasing.
  Stored text gets a folded column (NFKC + casefold) and query words are
  folded the same way, so case is ignored in every script.
- Prompt label "Records related to this question" (matches may now be recent);
  eval re-run 3x on qwen2.5:7b, all pass. Spec, memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:50:37 +09:00
javis-bot
54a465948a feat(memory): v2 conversation history store and its context in the reply prompt
- src/jarvis/memory/history.py keeps voice transcripts, Jarvis's answers,
  registered chat channel messages (with edits and deletes) and voice channel
  presence in one SQLite file. Every read is scoped to the asker: own records
  always, chat channels and others' words (only while the asker was in the
  same voice channel) behind settings, plus an optional since and off.
- Search is language-agnostic: Unicode words, prefixes keeping 60% of the
  word, rarer words weigh more, newer wins ties. 100k records search in
  24-61 ms here. context() returns the recent window and the matches cut to a
  token budget, matches at most half so chatter cannot push out the record
  the question is about.
- run_reply_engine(history=...) puts that context into the system prompt
  under "Information shared in previous conversations", fenced as data.
  evals/test_history_context_recall.py: with the block qwen2.5:7b answers the
  stored day in Korean and English (3 of 3 runs), without it the day is unknown.
- Spec history.spec.md, registry entry in CLAUDE.md, reply.spec.md,
  docs/llm_contexts.md, docs/v2/memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:45:33 +09:00
javis-bot
6b23a8adc7 feat(llm): talk to OpenAI-compatible servers such as Strata
A base URL that ends in /v1 now selects the OpenAI-compatible API: chat goes
to /chat/completions and embeddings to /embeddings. The conversion lives in
src/jarvis/llm.py only, so the reply engine and every other caller keep their
Ollama-shaped messages and responses.

- Requests drop private engine keys, send tool-call arguments as JSON strings,
  map num_predict to max_tokens and the thinking switch to
  chat_template_kwargs.enable_thinking, and leave out num_ctx and keep_alive.
- Responses come back with message.content, tool calls with dict arguments and
  an id, and message.thinking from reasoning_content.
- Streaming, the warm-up (one-token chat completion), the intent judge and the
  dictation clean-up use the same path. A server without an embedding model
  fails open to the text paths.
- Specs and docs/llm_contexts.md describe the transport; roadmap records M3
  step 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:32:48 +09:00
javis-bot
5efa760986 test(app): keep hearing a speaker across DAVE group changes, and record the Rust decision
- Live test: the speaker leaves and rejoins three times; each time its sentence is heard exactly
  under the new SSRC (songbird issue #310 reports receive stalls after group changes; not reproduced).
- decisions.md: Rust over Electron, with the side-by-side results, the songbird risk and the fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:03:41 +09:00
javis-bot
f25c26cf53 refactor(app): rewrite the M2 Discord voice core in Rust
The user asked for a faster, real app instead of Electron. The app is now a Rust crate (serenity +
songbird); the GUI in M5 will be a Tauri 2 shell on the system WebView2.

- Same behaviour as the Electron core: one voice channel, per-speaker PCM to the voice worker,
  speaking_end and speaker_leave, sentence playback with stop and cancel, transcription notice on
  join, slash command join/leave/stop (create-or-update), registered chat channels with live events,
  history and sends that never ping, worker process with a fresh token and restart backoff.
- songbird 0.6.1 garbles about a third of received frames under DAVE (its decoder keeps the bytes
  DAVE decryption leaves at the end of the payload; a 20 ms stepped test tone showed repeats, skips
  and reordering, while sending was exact). songbird now only decrypts; the core reorders packets in
  a small window (Opus FEC for the last lost frame) and decodes them with libopus itself.
- Live test on real Discord with two bots: five sentences heard (CER 0/0/0/5/12.5%), 0.71-0.90 s
  from end of speech to final, speech read back at 3.4%, sound gone 0.08 s after stop. The release
  binary uses about 16 MB of memory; a Windows build (MinGW) logs in under Wine.
- Spec moved to app/src/discord_core.spec.md; architecture, decisions, roadmap and pc-features updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:57:53 +09:00
javis-bot
8479112ddd feat(app): M2 Discord voice core in the Electron app
- app/ (Electron 45 + TypeScript): the core joins one voice channel, decodes each speaker's Opus to
  48 kHz stereo and streams it to the voice worker, sends speaking_end and speaker_leave, and emits
  speech_start/partial/final with the speaker's display name.
- Sentence playback from the worker's TTS, first come first served, with stop that silences the clip
  at once and cancels unfinished requests; slash command join/leave/stop (create-or-update by name).
- Transcription notice posted on every join; other bots ignored by default; at most maxSpeakers
  people transcribed at once.
- Registered chat channels: live message events, history reads, sends split at 2,000 characters with
  mentions disabled; Message Content intent checked before login.
- The app starts the Python voice worker with a fresh token in the environment, restarts it with
  backoff, and the worker client reconnects and fails in-flight TTS on a lost connection.
- Spec in app/src/core/discord_core.spec.md; Vitest unit tests, a protocol test against the real
  Python worker server with fake engines, and an opt-in live test with two bots on real Discord.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:14:25 +09:00
javis-bot
1309525aa8 fix(voice_worker): the wake word hint may never add, drop or create a wake word
Hotword occurrences were counted with fuzzy matching, which never counted a one-letter wake word
(so a hinted "봇 ..." was accepted over a plain decode without it) and counted the real word
"서비스" as "자비스" (so the hint could turn it into the wake word). Occurrences are now counted
exactly (case, spacing and punctuation ignored); near misses are left as heard for call detection.
The real-noise eval now plays each recording as recorded and at -25 dBFS and reports how many clips
reached the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:40:42 +09:00
javis-bot
590cd07268 fix(voice_worker): even out speaker loudness and stop transcribing noise into words
- Loudness equalisation replaces peak normalisation: rumble below 60 Hz removed, level measured on
  the VAD-voiced windows only, brought to -20 dBFS (gain -20..+40 dB), soft limiter below 0.98.
- Only clearly voiced windows (VAD >= 0.5) count towards the 250 ms needed before an utterance
  reaches the model, so noise the VAD half-believes never calls Whisper.
- Hallucination gate: mean token log-probability >= -0.7 (the old len/(len+1) correction let short
  hallucinations through), compression ratio <= 2.4; no_speech_prob dropped (always 0.0 on turbo).
- With hotwords, the plain decode of the same encoder pass decides; the hint may fix a wake word's
  spelling but cannot add or drop one (it wrote the wake word on laughter and after real sentences).
- Opt-in real-model evals: voice-like babble with the hint, speech over babble, quiet and clipped
  speakers, and a JARVIS_NOISE_DIR folder of real noise recordings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:27:38 +09:00
javis-bot
a7685e9078 feat(voice_worker): M1 multi-speaker streaming STT and sentence TTS over a local WebSocket
- One shared faster-whisper model for every speaker, batched; finals go before partials and
  partial batches are capped at two so a final never waits behind a ten-speaker pass.
- Per-speaker 48k stereo to 16k mono conversion, streaming Silero VAD, utterance ends on VAD
  silence, Discord speaking_end, missing audio, or the 28 s Whisper window.
- MeloTTS in its own venv through a child process; sentences stream out one chunk each, with
  incremental text, cancel and failure reporting.
- Loopback-only server with a token from the environment; logs carry no transcript text.
- Spec in src/voice_worker/voice_worker.spec.md; unit tests with fake engines and an opt-in
  real-GPU test that streams ten speakers at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:00:06 +09:00
javis-bot
b4926d5e19 docs(v2): note that the game went behind the kit's console because the user clicked it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:36:11 +09:00