fix(app): the LLM placement measures only what it can trust and keeps a game the 0 GiB step

- a measurement whose short answer fails, or whose second card reading fails or shows nothing
  more held, is not kept: the model starts with expert-vram=0 and the next start measures again
- measurements are kept per context and requests at once (<context>/<parallel>) and per card
  (its NVIDIA UUID), not by the card's size
- a server on this app's port left from a run that ended badly is stopped before the card is
  read, so its memory is not counted as other programs'
- the game placement falls back to 0 GiB of experts before pausing when 2 GiB leaves a game
  nothing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-11-14 10:01:53 +09:00
parent af6f2c855d
commit 1400a3ff97
4 changed files with 230 additions and 59 deletions

View File

@@ -386,10 +386,16 @@ start it first, so nothing answers before the brain can, and stop it last.
experts, the STT worker's share (2740 MiB: its model and ten speakers at once, M0 `stt`), the
engine's margin (768 MiB) and the room for programs opened later (`game.otherRoomGb`, default
1 GB); the budget first holds the long-prompt staging, rounded down to 0.1 GiB, 0 when nothing is
left. What the model holds without experts is measured once per context and card (the first-run
self-check: a start with `expert-vram=0`, one short answer, then the card's memory again and the
staging from the engine log), kept in `config/llm-probe.json`; the real start then replaces that
server. Without a card nothing is set. The `llm` log tab gets the placement and what it came from.
left. What the model holds without experts is measured once per context, number of requests at
once and card (its NVIDIA UUID): the first-run self-check, a start with `expert-vram=0`, one
short answer, then the card's memory again and the staging from the engine log; kept in
`config/llm-probe.json` under `<context>/<parallel>`, and the real start then replaces that
server. A measurement whose answer or second reading fails (or shows nothing more held) is not
kept: the model starts with `expert-vram=0` and the next start measures again. A server of this
app's port that still answers before the start (left from a run that ended badly) is stopped
first, so it is not counted as other programs' memory. Without a card nothing is set. The game
placement (used by the switching that comes next) takes the reserve when it fits beside more
than 2 GiB of experts, else 2 GiB and the room that leaves, else 0 GiB and its room, else pause. The `llm` log tab gets the placement and what it came from.
- **A server the app does not start**: with `llm.url` (an OpenAI-compatible `/v1` address, in the
settings file only) the app starts nothing; that server must list a model within 30 seconds.
- **Settings**: `llm.context` (default 32768 tokens per request, 4096-262144) and `llm.port`