- screenView tool (screen_view.spec.md): whole screen, monitor by number, window by part of its
title or program, or the list of monitors and windows; fast, normal or detailed pictures.
- The app captures with Windows Graphics Capture (app/src/screen.rs, admin.spec.md "Screen
viewing"), decides for the asker (only people registered for PC control), never picks, lists
or shows excluded programs (pcControl.screenExclude, password managers by default), refuses
minimised windows and black pictures, and records screen.view in the activity log.
- Pictures ride on the tool message (ToolExecutionResult.images); llm.py sends them to
OpenAI-compatible servers as image parts after the tool messages. They stay in that reply only:
not in the tool carryover, logs or prompt dumps.
- The LLM reads pictures only when screen viewing is available (--vision gpu normally, cpu in a
game), measured apart for the placement; otherwise screenView is a disabled tool.
- Worker: screen_action / screen_action_result, 4 MiB messages.
- Eval evals/test_screen_view_tool.py; Windows check tests/screen_windows.rs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- a normal placement keeps its reading only when no game runs: the perf loop passes whether a
game runs to every switch, so the way back from a failed game stage while the game runs keeps
the reading from before the game
- game detection also looks at every running program once as the services start: a program in
a launcher's game folder or the person's games list counts, also behind other windows
- checked on the win-ci VM: a minimised program in a Steam game folder is found at the start
while nothing is in front
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- the game stage reads the card again: other programs as at the last normal placement made with
no game running, and a reserve of at least what grew since (the game and programs opened after
that reading); with a game already running at the start (game detection now looks once before
the LLM starts) it counts what the card holds now and leaves only the room for other programs,
so it no longer pauses for counting the game twice
- the retry count starts over whenever another stage is wanted, also when the game ends with the
normal stage already running, so the next game tries again
- the wait before the next try counts from when the failed switch and its way back are over
- the worker client keeps, sends and readies a connection under one lock (llm, then state), so an
LLM state set while a worker connects is not lost
- a game watch dropped by a start that fails part way stops its thread
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- the normal stage reads other programs' memory afresh (the card after the LLM stopped, less the
STT worker's model) and keeps it; the game stage counts from the latest normal reading, so the
game is not counted against its own reserve
- a restart that fails brings back the stage that ran before and tells the brain ready; the
failed stage is tried again after 60 s, three times at most; when nothing comes back the brain
is told paused so requests fail at once
- the worker client keeps the latest llm state and sends it first on every connection
- switches are numbered: llm switching carries an id, llm_idle names it, and a late idle of an
earlier switch no longer lets a new one through
- a request that waited 180 s for a switch fails without calling the brain
- the dashboard shows the stage that runs also on a fallback placement without experts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- while the services run, game mode decides the stage (perf.rs): a game means the game stage at
once (or pause with game.pauseInGame); its end brings the normal stage back after
game.revertSeconds (default 30)
- a change tells the voice worker the LLM switches (llm switching, with a notice), waits until
the brain has written the answers in progress (llm_idle, at most 2 minutes), restarts the LLM in
the stage and tells the worker it is ready or paused
- the game stage runs at 16K context, one request at a time, half the engine's CPU threads and
the game placement (reserve game.reserveGb, default 8, else 2 GiB, else 0 GiB, else pause),
measured once at that context; every placement of a run counts other programs' memory as it
was when the services started
- the voice worker gates every LLM use: on switching new work waits (at most 180 s), the first
spoken request that waits hears the notice once under its own id, on ready all go on, on
paused nothing is answered
- dashboard: "성능 단계: 게임 중 · 게임에 X GB" / 일시정지; game mode screen: game VRAM, revert
wait, pause in game
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- strata launch --stop now always runs before the card is read and the start's own port must
refuse connections; a left server that answers, still loads (503) or stays silent was skipped
before and its memory counted as other programs'
- a stop that is not confirmed reads and keeps nothing: the model starts with expert-vram=0
- the stop confirms on the port it is given (stop_server_on), not on one remembered from an
earlier start
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- a measurement whose short answer fails, or whose second card reading fails or shows nothing
more held, is not kept: the model starts with expert-vram=0 and the next start measures again
- measurements are kept per context and requests at once (<context>/<parallel>) and per card
(its NVIDIA UUID), not by the card's size
- a server on this app's port left from a run that ended badly is stopped before the card is
read, so its memory is not counted as other programs'
- the game placement falls back to 0 GiB of experts before pausing when 2 GiB leaves a game
nothing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- with an NVIDIA card the app sets --expert-vram itself by the M0 kit's formula: what the card
holds before the start, the model without experts, the STT worker's share (2740 MiB), the
engine's margin (768 MiB) and room for programs opened later (game.otherRoomGb, default 1 GB)
are left free; Strata's own placement filled the card and left the STT no room
- the model without experts is measured once per context and card (a start with
expert-vram=0, one short answer, the card's memory and the engine log's staging), kept in
config/llm-probe.json; the real start replaces that server
- placement.rs also works out the game placement (the reserve, else the 2 GiB minimum and the
room a game really gets, else pause) for the switching that comes next
- with the game PC's M0 numbers the normal placement comes out at 1.5 GiB, as M0 measured
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Settings `features` (weather, webSearch, meals) and `mcpServers` (name, command, args, env names,
enabled), edited on the new 기능 screen. The tools of a feature that is off go into the brain's
`disabled_tools`; only enabled MCP servers go into its `mcps`.
- MCP environment values (API keys) live only in the encrypted secrets (`mcp:<server>:<NAME>`),
shown as stored or not, masked in logs, removed with their server, and reach the worker only in
`JARVIS_MCP_ENV`, which the brain takes out of its environment before giving each server its own.
- `enabled_builtin_tools(cfg)` is the one list the router, toolSearchTool and the executor use, so
a turned-off tool is neither offered nor run, also when the model names it.
- The v2 brain never offers v1's `localFiles` (home folder read/write/delete) and `screenshot`:
anyone in the voice or chat channels could ask for them. Screen viewing comes in M7.
- The brain connects MCP servers on its own thread and logs each outcome without debug logging.
- Stored secrets are masked in the log hub as soon as the data folder is found.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups to M5.
- Chat channel names and topics (app/src/channel_sync.rs): a channel with
syncDiscord (default on) gets its name and description as its Discord
name and topic when saved; registering fills them from Discord;
Discord's form of the name is written back; a topic over 1,024
characters is refused before saving; a rate-limited edit is pending and
retried once by itself with the latest text; a missing permission fails
with the reason and the app keeps the text. Commands
discord_channel_info and channel_sync_status. Checked on real Discord
with a temporary channel (fill, edit, rate limit, automatic retry).
- voice.tts.speed (0.5-2.0) on the model screen, passed to the worker as
--melo-speed. MeloTTS's Korean model has one voice, so no voice choice.
- Closing the window runs AdminApi::shutdown first: an install in
progress is cancelled and waited for, the services stop, a failed run's
leftovers (Strata) are cleared (Controller::shutdown). If that fails the
window stays open with the reason; closing again quits.
- WebView2 data goes in the data folder (cache/webview), or a temp scratch
folder before one is chosen, removed at the next start: %APPDATA%\JarvisBot
holds only the location file.
- Command output that is not UTF-8 (CP949 on a PC without the UTF-8
option) is decoded in the OEM code page instead of ending the reading.
Verified on the win-ci VM: closing during the install leaves no process,
no --location keeps only location.json under %APPDATA%, the unit tests on
Windows 168/168.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found running the app's admin commands on the game PC (Windows, RTX 5070
Ti): the full install, Strata started by the app downloading the 22.9 GB
model, the worker and Discord up, five spoken questions all answered.
- Windows refuses a connection to a closed local port only after about
2 s, so the 2 s request timeout made every stop "not confirmed". The
check now waits up to 10 s to connect and still counts only a refusal
(not a timeout) as stopped.
- Strata 0.1.59 reports the model download as "X of Y GB"; percent()
reads that, and the dashboard shows the current line's progress only
(it had stayed on the engine download's 80 %).
- Strata and the NVIDIA shader cache write under %APPDATA% whatever
STRATA_* says: the layout sets APPDATA inside the data folder on
Windows, next to LOCALAPPDATA.
Roadmap: M5 step 5 done.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
M5 step 5, first part. app/shell is a Tauri 2 crate (jarvis.exe): one
window over app/ui whose invoke("admin") goes to AdminApi::call. Its
WebView2 data sits next to the location file; closing the window stops
the services and waits for a cancelled start (AdminApi::shutdown).
app/shell/package.sh cross-builds it on Linux (x86_64-pc-windows-gnu)
and lays out Jarvis/ with WebView2Loader.dll and python-src/
(voice_worker, jarvis). The worker's default sources are python-src next
to the executable when present, else the repository's src/.
The bot screen now shows the masked token right after saving.
Checked on the win-ci VM: the window and wizard, PC check, uv and worker
venv installed, a cancel leaving no uv/Python process, the token stored
DPAPI-encrypted, a start failing with its reason, logs, error copy, zip
export, closing the window ending the process, nothing under %APPDATA% or
%LOCALAPPDATA%; the core's unit tests on Windows (156/157, the one
Linux-path assumption fixed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stop right after a background start set the LLM's cancel flag, but the
start task then cleared it in Llm::start and went on to download and load
the model. The cancel is now armed in the controller's begin_start, under
its lock (Host::prepare_start -> Llm::arm), and Llm::start never clears
it; a start that finds it set returns before running anything.
wait_gone counted a timeout, a 503 (a server still loading) or an
unreadable answer as stopped, so a failed `launch --stop` could still mark
a live engine off. Only a refused connection confirms the stop now; the
rest keeps checking until the deadline and then reports failed.
Tests: a stop right after a background start runs no launch (and the next
start is not cancelled); a failed stop with a silent or 503 server leaves
the LLM and the services failed through the controller. Fixed-port tests
use an unused port.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
service_start now moves the controller to starting before it returns
(Controller::start_in_background), so a stop right after it cancels the
start and a second start is refused instead of the start running late.
service_restart stops, then starts the same way.
Stopping Strata now confirms the server is gone: `launch --stop` is ended
after 2 minutes with everything it started, then the port must stop
listing a model. A server still answering makes the LLM failed with the
reason, and Services::stop / Running::stop return it, so the controller
shows failed instead of stopped (also for a cancelled start that could
not clear the server). A stop command that fails while nothing answers
counts as stopped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
M5 step 4, second part. The services start the LLM first: the installed
Strata with the M0 kit's model and normal placement (models in
models/llm, parallel 1, or 2 when a chat channel answers every message,
pictures off, --restart), wait until /v1/models lists the model, and stop
it last. The first start downloads the model; the dashboard shows
Strata's last line and percentage, and stop also cancels a start (the
server is cleared). llm.url uses a server the app does not start.
On every start the app writes config/brain.json (Strata's /v1, the
listed model id, memory/jarvis.db) and hands it to the worker as
JARVIS_CONFIG_PATH. Loading a config with its own db_path no longer
creates ~/.local/share/jarvis.
service_start/service_restart run in the background. Settings gain
llm.context and llm.port (model screen) and llm.url (file only).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>