fix(app): the LLM placement measures only what it can trust and keeps a game the 0 GiB step
- a measurement whose short answer fails, or whose second card reading fails or shows nothing more held, is not kept: the model starts with expert-vram=0 and the next start measures again - measurements are kept per context and requests at once (<context>/<parallel>) and per card (its NVIDIA UUID), not by the card's size - a server on this app's port left from a run that ended badly is stopped before the card is read, so its memory is not counted as other programs' - the game placement falls back to 0 GiB of experts before pausing when 2 GiB leaves a game nothing Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -386,10 +386,16 @@ start it first, so nothing answers before the brain can, and stop it last.
|
||||
experts, the STT worker's share (2740 MiB: its model and ten speakers at once, M0 `stt`), the
|
||||
engine's margin (768 MiB) and the room for programs opened later (`game.otherRoomGb`, default
|
||||
1 GB); the budget first holds the long-prompt staging, rounded down to 0.1 GiB, 0 when nothing is
|
||||
left. What the model holds without experts is measured once per context and card (the first-run
|
||||
self-check: a start with `expert-vram=0`, one short answer, then the card's memory again and the
|
||||
staging from the engine log), kept in `config/llm-probe.json`; the real start then replaces that
|
||||
server. Without a card nothing is set. The `llm` log tab gets the placement and what it came from.
|
||||
left. What the model holds without experts is measured once per context, number of requests at
|
||||
once and card (its NVIDIA UUID): the first-run self-check, a start with `expert-vram=0`, one
|
||||
short answer, then the card's memory again and the staging from the engine log; kept in
|
||||
`config/llm-probe.json` under `<context>/<parallel>`, and the real start then replaces that
|
||||
server. A measurement whose answer or second reading fails (or shows nothing more held) is not
|
||||
kept: the model starts with `expert-vram=0` and the next start measures again. A server of this
|
||||
app's port that still answers before the start (left from a run that ended badly) is stopped
|
||||
first, so it is not counted as other programs' memory. Without a card nothing is set. The game
|
||||
placement (used by the switching that comes next) takes the reserve when it fits beside more
|
||||
than 2 GiB of experts, else 2 GiB and the room that leaves, else 0 GiB and its room, else pause. The `llm` log tab gets the placement and what it came from.
|
||||
- **A server the app does not start**: with `llm.url` (an OpenAI-compatible `/v1` address, in the
|
||||
settings file only) the app starts nothing; that server must list a model within 30 seconds.
|
||||
- **Settings**: `llm.context` (default 32768 tokens per request, 4096-262144) and `llm.port`
|
||||
|
||||
Reference in New Issue
Block a user