feat(m0): measure the 1 GB room for other programs and a 95 % GPU cap
User decision (2026-11-10, room to watch a video next to Jarvis): the normal placement leaves 1 GB of VRAM for programs opened later, and Jarvis's GPU work is capped at 95 %. - gamemode_tune's normal rows take OTHER_ROOM_MIB off the expert budget. - gpucap.DutyCap pauses the Strata engine (strata.exe --serve under the kit) and the STT worker with their children together for 10 ms of every 200 ms (NtSuspendProcess), and always resumes them. - Every row samples nvidia-smi utilization.gpu; capped rows have an uncapped twin (normal with and without the STT, game 2 GB with 2 s partials) so the run shows the cost and what the GPU really gets back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -68,7 +68,7 @@ LLM 단계는 8080 포트를 씁니다. 다른 프로그램이 이 포트를 쓰
|
||||
|
||||
`run_m0_gamemode.bat`을 더블클릭하면 게임 중 자동 조절 단계 두 개(`gamemode`, `gamemode_stt`)만 돕니다. 먼저 LLM만 배치 5개(평소, 게임 중 후보 3개, 다시 평소)로 재고, 이어서 음성 인식을 함께 켠 배치 6개를 잽니다. 모델은 이미 받아 둔 것을 쓰므로 새로 받지 않고(음성 인식 모델을 아직 안 받았으면 약 1.6GB를 받습니다), 20~30분 걸립니다. 그동안 게임이나 무거운 프로그램은 꺼 두시고 PC에 손대지 않으시면 됩니다.
|
||||
|
||||
`run_m0_gamemode_tune.bat`을 더블클릭하면 `gamemode_tune` 단계만 돕니다. 앞 측정에서 정한 두 가지를 확인합니다. (1) 평소 배치에 음성 인식(STT) 자리를 남겼을 때의 속도와 첫 토큰 시간입니다. LLM을 전문가 층 없이 한 번 켜서 기본 VRAM을 재고, STT가 10명 동시 전사 때 더 쓰는 VRAM과 여유분을 뺀 나머지를 전문가 층에 줍니다. (2) 게임 중 배치(전문가 2GB)에서 LLM이 답하는 동안 부분 전사를 2초마다 하거나 멈추고 최종 전사만 바로 할 때, LLM 속도와 최종 전사가 기다린 시간입니다. 게임이 없을 때는 부분 전사를 줄이지 않으므로 평소 배치는 이 비교를 하지 않습니다. 게임은 켜지 않고 잽니다. 이미 받은 모델을 쓰고 10~15분 걸립니다. 그동안 게임이나 무거운 프로그램은 꺼 두시고 PC에 손대지 않으시면 됩니다.
|
||||
`run_m0_gamemode_tune.bat`을 더블클릭하면 `gamemode_tune` 단계만 돕니다. 앞 측정에서 정한 세 가지를 확인합니다. (1) 평소 배치에 음성 인식(STT) 자리와 다른 프로그램용 여유 VRAM 1GB를 남겼을 때의 속도와 첫 토큰 시간입니다. LLM을 전문가 층 없이 한 번 켜서 기본 VRAM을 재고, STT가 10명 동시 전사 때 더 쓰는 VRAM, 여유분, 다른 프로그램용 1GB를 뺀 나머지를 전문가 층에 줍니다. (2) 게임 중 배치(전문가 2GB)에서 LLM이 답하는 동안 부분 전사를 2초마다 하거나 멈추고 최종 전사만 바로 할 때, LLM 속도와 최종 전사가 기다린 시간입니다. 게임이 없을 때는 부분 전사를 줄이지 않으므로 평소 배치는 이 비교를 하지 않습니다. (3) GPU 사용률 95% 제한입니다. LLM 엔진과 STT를 200ms마다 함께 10ms씩 멈추고, 제한이 없을 때와 비교해 속도와 실제 GPU 사용률을 잽니다. 이때 두 프로그램이 잠깐씩 멈추는 것은 정상입니다. 게임은 켜지 않고 잽니다. 이미 받은 모델을 쓰고 20~30분 걸립니다. 그동안 게임이나 무거운 프로그램은 꺼 두시고 PC에 손대지 않으시면 됩니다.
|
||||
|
||||
`run_m0_game_live.bat`을 더블클릭하면 실제 게임을 켠 상태로 잽니다(`gamemode_live`). 게임을 창 모드로 켜고 플레이 화면까지 들어간 뒤 Enter를 누르고 15초 안에 게임으로 돌아가면 됩니다. 게임 중 배치 4개(전문가 2GB·0GB, 음성 인식 GPU·CPU)를 차례로 재고 10~15분 걸리며, 끝나면 알림음이 울립니다. 게임 화면이나 게임 이름은 저장하지 않고 VRAM 사용량과 속도만 적습니다. 그래픽 메모리를 많이 쓰는 게임이면 배치가 켜지지 않을 수 있는데, 그것도 결과로 적습니다.
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ STAGES = [
|
||||
["model=fast", "only=together"], 0),
|
||||
# The fixes for what gamemode_stt showed: room for the STT in the normal placement, and fewer
|
||||
# STT partial passes while the LLM answers (normal and the 2 GB game placement).
|
||||
("gamemode_tune", "게임 중 자동 조절: STT 자리 남기기와 부분 전사 간격", "base", "llm.py",
|
||||
("gamemode_tune", "자동 조절: STT 자리와 VRAM 여유 1GB, 부분 전사 간격, GPU 95% 제한", "base", "llm.py",
|
||||
["model=fast", "only=tune"], 0),
|
||||
# With the user's game running. Needs someone at the PC, so only when asked for (--only).
|
||||
("gamemode_live", "게임을 켠 상태로 재기 (직접 게임을 켜 주셔야 합니다)", "base", "llm.py",
|
||||
|
||||
201
scripts/m0/m0lib/gpucap.py
Normal file
201
scripts/m0/m0lib/gpucap.py
Normal file
@@ -0,0 +1,201 @@
|
||||
"""The GPU share cap the app would put on Jarvis (user decision, 2026-11-10: 95 %).
|
||||
|
||||
Jarvis's GPU work runs in two processes: the Strata engine and the voice worker (here the STT
|
||||
load worker). A GPU cannot be told to give one process only part of its time, so the cap pauses
|
||||
both processes together for a short slice of every period (Windows NtSuspendProcess /
|
||||
NtResumeProcess): 10 ms of every 200 ms is 95 %. Work already queued on the GPU still finishes
|
||||
during the pause, so what the GPU really gets back is measured (``UtilSampler``), not assumed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Callable, Iterable
|
||||
|
||||
from m0lib.common import IS_WINDOWS, nvidia_smi, stats
|
||||
|
||||
PERIOD_S = 0.2
|
||||
|
||||
|
||||
def gpu_util_pct() -> int | None:
|
||||
"""nvidia-smi's utilization.gpu of GPU 0: the share of the last sample period in which any
|
||||
kernel ran (what the app would read through NVML), or None."""
|
||||
exe = nvidia_smi()
|
||||
if not exe:
|
||||
return None
|
||||
try:
|
||||
out = subprocess.run([exe, "--query-gpu=utilization.gpu", "--format=csv,noheader,nounits"],
|
||||
capture_output=True, text=True, timeout=10).stdout.strip().splitlines()[0]
|
||||
return int(out.strip())
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
class UtilSampler:
|
||||
"""Samples GPU utilisation in the background while a measurement runs."""
|
||||
|
||||
def __init__(self, interval: float = 0.5, read: Callable[[], int | None] | None = None):
|
||||
self.interval = interval
|
||||
self.read = read or gpu_util_pct
|
||||
self.samples: list[int] = []
|
||||
self._stop = threading.Event()
|
||||
self._t: threading.Thread | None = None
|
||||
|
||||
def __enter__(self) -> "UtilSampler":
|
||||
self._t = threading.Thread(target=self._loop, daemon=True)
|
||||
self._t.start()
|
||||
return self
|
||||
|
||||
def _loop(self) -> None:
|
||||
while not self._stop.is_set():
|
||||
v = self.read()
|
||||
if v is not None:
|
||||
self.samples.append(v)
|
||||
self._stop.wait(self.interval)
|
||||
|
||||
def __exit__(self, *exc) -> None:
|
||||
self._stop.set()
|
||||
if self._t:
|
||||
self._t.join(timeout=15)
|
||||
|
||||
def summary(self, cap: float | None = None) -> dict:
|
||||
out = {"pct": stats(self.samples, 1)}
|
||||
if cap is not None and self.samples:
|
||||
out["over_cap_share"] = round(sum(1 for v in self.samples if v > cap * 100) / len(self.samples), 3)
|
||||
return out
|
||||
|
||||
|
||||
def process_table() -> list[dict]:
|
||||
"""Every process with its id, parent, executable and command line (Windows only)."""
|
||||
if not IS_WINDOWS:
|
||||
return []
|
||||
ps = ("Get-CimInstance Win32_Process | ForEach-Object { \"$($_.ProcessId)`t$($_.ParentProcessId)`t"
|
||||
"$($_.ExecutablePath)`t$($_.CommandLine)\" }")
|
||||
try:
|
||||
out = subprocess.run(["powershell", "-NoProfile", "-Command", ps], capture_output=True, text=True,
|
||||
encoding="utf-8", errors="replace", timeout=60).stdout
|
||||
except Exception:
|
||||
return []
|
||||
return parse_process_table(out)
|
||||
|
||||
|
||||
def parse_process_table(text: str) -> list[dict]:
|
||||
rows = []
|
||||
for line in text.splitlines():
|
||||
parts = line.rstrip("\r").split("\t", 3)
|
||||
if len(parts) < 4 or not parts[0].strip().isdigit():
|
||||
continue
|
||||
rows.append({"pid": int(parts[0]), "ppid": int(parts[1]) if parts[1].strip().isdigit() else None,
|
||||
"exe": parts[2], "cmd": parts[3]})
|
||||
return rows
|
||||
|
||||
|
||||
def with_children(procs: list[dict], roots: Iterable[int]) -> list[int]:
|
||||
"""The roots and everything they started (a venv python.exe on Windows is a launcher that
|
||||
runs the real interpreter as its child)."""
|
||||
out, todo = [], [r for r in roots if r is not None]
|
||||
while todo:
|
||||
pid = todo.pop()
|
||||
if pid in out:
|
||||
continue
|
||||
out.append(pid)
|
||||
todo += [p["pid"] for p in procs if p["ppid"] == pid]
|
||||
return out
|
||||
|
||||
|
||||
def engine_roots(procs: list[dict], home: Path) -> list[int]:
|
||||
"""The Strata engine serving the model: its strata.exe --serve under our components folder
|
||||
(not the strata terminal that started it)."""
|
||||
where = str(home / "components").lower()
|
||||
return [p["pid"] for p in procs
|
||||
if p["exe"].lower().startswith(where) and p["exe"].lower().endswith("strata.exe")
|
||||
and "--serve" in p["cmd"]]
|
||||
|
||||
|
||||
_API: dict = {}
|
||||
|
||||
|
||||
def _nt(name: str, pid: int) -> bool:
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
if not _API:
|
||||
k32 = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
ntdll = ctypes.WinDLL("ntdll")
|
||||
k32.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD]
|
||||
k32.OpenProcess.restype = ctypes.c_void_p
|
||||
k32.CloseHandle.argtypes = [ctypes.c_void_p]
|
||||
for fn in (ntdll.NtSuspendProcess, ntdll.NtResumeProcess):
|
||||
fn.argtypes = [ctypes.c_void_p]
|
||||
fn.restype = ctypes.c_long
|
||||
_API.update(k32=k32, ntdll=ntdll)
|
||||
k32 = _API["k32"]
|
||||
h = k32.OpenProcess(0x0800, False, pid) # PROCESS_SUSPEND_RESUME
|
||||
if not h:
|
||||
return False
|
||||
try:
|
||||
return getattr(_API["ntdll"], name)(h) == 0
|
||||
finally:
|
||||
k32.CloseHandle(h)
|
||||
|
||||
|
||||
def suspend_pid(pid: int) -> bool:
|
||||
return _nt("NtSuspendProcess", pid)
|
||||
|
||||
|
||||
def resume_pid(pid: int) -> bool:
|
||||
return _nt("NtResumeProcess", pid)
|
||||
|
||||
|
||||
class DutyCap:
|
||||
"""Pause ``pids`` together for (1 - cap) of every ``period`` s while the block runs; every
|
||||
paused process is resumed before the block ends, also on an error."""
|
||||
|
||||
def __init__(self, pids: list[int], cap: float, period: float | None = None,
|
||||
suspend: Callable[[int], bool] | None = None, resume: Callable[[int], bool] | None = None):
|
||||
self.pids, self.cap, self.period = list(pids), cap, period or PERIOD_S
|
||||
self.suspend, self.resume = suspend or suspend_pid, resume or resume_pid
|
||||
self.pause = self.period * (1 - cap)
|
||||
self.paused_s = 0.0
|
||||
self.cycles = 0
|
||||
self.failed: set[int] = set()
|
||||
self._stop = threading.Event()
|
||||
self._t: threading.Thread | None = None
|
||||
self._t0 = 0.0
|
||||
self._t1 = 0.0
|
||||
|
||||
def __enter__(self) -> "DutyCap":
|
||||
self._t0 = time.perf_counter()
|
||||
if self.pids and self.pause > 0:
|
||||
self._t = threading.Thread(target=self._loop, daemon=True)
|
||||
self._t.start()
|
||||
return self
|
||||
|
||||
def _loop(self) -> None:
|
||||
while not self._stop.wait(self.period - self.pause):
|
||||
held = [p for p in self.pids if self.suspend(p)]
|
||||
self.failed.update(p for p in self.pids if p not in held)
|
||||
s0 = time.perf_counter()
|
||||
try:
|
||||
time.sleep(self.pause)
|
||||
finally:
|
||||
for p in held:
|
||||
if not self.resume(p):
|
||||
self.failed.add(p)
|
||||
self.paused_s += time.perf_counter() - s0
|
||||
self.cycles += 1
|
||||
|
||||
def __exit__(self, *exc) -> None:
|
||||
self._stop.set()
|
||||
if self._t:
|
||||
self._t.join(timeout=10)
|
||||
self._t1 = time.perf_counter()
|
||||
|
||||
def summary(self) -> dict:
|
||||
span = (self._t1 or time.perf_counter()) - self._t0
|
||||
return {"cap": self.cap, "period_s": self.period, "pause_s": round(self.pause, 4),
|
||||
"processes": len(self.pids), "cycles": self.cycles,
|
||||
"paused_share": round(self.paused_s / span, 3) if span > 0 else None,
|
||||
"failed_processes": len(self.failed)}
|
||||
@@ -10,8 +10,9 @@ Options (--opt key=value):
|
||||
only=together the same with the STT loaded next to it and transcribing ten
|
||||
speakers at once, on the GPU and on the CPU (needs stt_python=)
|
||||
only=live game placements and the STT while the user runs a real game
|
||||
only=tune the normal placement with room left for the STT, and in the game
|
||||
placement the STT doing fewer partial passes while the LLM answers
|
||||
only=tune the normal placement with room left for the STT and 1 GB for other
|
||||
programs, the game placement with the STT doing fewer partial passes
|
||||
while the LLM answers, and both with Jarvis capped at 95 % of the GPU
|
||||
stt_python=PATH the Python of the stt venv (m0.py passes it)
|
||||
"""
|
||||
|
||||
@@ -34,6 +35,7 @@ from m0lib.common import (IS_WINDOWS, GpuSampler, StageResult, ask, gpu_memory_m
|
||||
from m0lib.llm_bench import (bench_single, quick_bench, test_decode, test_images, test_parallel, # noqa: E402
|
||||
test_tools, test_vram_shrink, warmup)
|
||||
from m0lib.tool_fix import test_tools_fixed, tools_passed # noqa: E402
|
||||
from m0lib import gpucap # noqa: E402
|
||||
from m0lib.llm_client import ChatClient, http_json # noqa: E402
|
||||
|
||||
PRESETS = {
|
||||
@@ -111,18 +113,30 @@ LIVE = [lv for lv in TOGETHER if lv["key"].startswith("game_")]
|
||||
# whole run. Without a game the STT keeps every partial (user decision), so only the game
|
||||
# placement is paced; its back-to-back row is the baseline measured in the same run.
|
||||
# The 2 GB expert placement is the game minimum on a 16 GB card (faster than 0 GB in a real game).
|
||||
# Then (user decision, 2026-11-10, for watching a video next to Jarvis):
|
||||
# - the normal placement also leaves OTHER_ROOM_MIB free for programs opened later ("other_room_mib");
|
||||
# - Jarvis's GPU work (the engine and the STT worker together) is paused for 5 % of every 200 ms
|
||||
# ("gpu_cap", gpucap.DutyCap). The rows with and without it measure what it costs and how much
|
||||
# GPU time it really gives back (queued work still runs during the pause).
|
||||
STT_ROOM_MARGIN_MIB = 768 # what Strata's qwen36 engine itself keeps free when it places experts
|
||||
STT_GROWTH_FALLBACK_MIB = 1440 # M0 stt on the game PC: ten speakers at once over the loaded model
|
||||
OTHER_ROOM_MIB = 1024 # the app's default room for other programs in the normal placement
|
||||
GPU_CAP = 0.95
|
||||
_PARTIAL_2S = {"partial_every": 2.0, "final_every": 0.5}
|
||||
_FINALS_ONLY = {"partial_every": None, "final_every": 0.5}
|
||||
_ROOM = {"parallel": 1, "vision": "gpu", "sets": ["prefill=2048"], "stt": "cuda", "stt_room": True}
|
||||
_ROOM = {"parallel": 1, "vision": "gpu", "sets": ["prefill=2048"], "stt_room": True,
|
||||
"other_room_mib": OTHER_ROOM_MIB}
|
||||
_GAME2 = {**_GAME, "context": 16384, "sets": _EXP2, "stt": "cuda"}
|
||||
TUNED = [
|
||||
{"key": "normal_stt_room_stt_gpu", **_ROOM},
|
||||
{"key": "game_experts_2g_16k_stt_gpu", **_GAME, "context": 16384, "sets": _EXP2, "stt": "cuda"},
|
||||
{"key": "game_experts_2g_16k_partial_2s", **_GAME, "context": 16384, "sets": _EXP2, "stt": "cuda",
|
||||
"stt_pace": _PARTIAL_2S},
|
||||
{"key": "game_experts_2g_16k_finals_only", **_GAME, "context": 16384, "sets": _EXP2, "stt": "cuda",
|
||||
"stt_pace": _FINALS_ONLY},
|
||||
# The room is worked out in the first row, with the STT loaded; the rows after it reuse it.
|
||||
{"key": "normal_room_stt_gpu", **_ROOM, "stt": "cuda"},
|
||||
{"key": "normal_room_stt_gpu_cap95", **_ROOM, "stt": "cuda", "gpu_cap": GPU_CAP},
|
||||
{"key": "normal_room_llm_only", **_ROOM},
|
||||
{"key": "normal_room_llm_only_cap95", **_ROOM, "gpu_cap": GPU_CAP},
|
||||
{"key": "game_experts_2g_16k_stt_gpu", **_GAME2},
|
||||
{"key": "game_experts_2g_16k_partial_2s", **_GAME2, "stt_pace": _PARTIAL_2S},
|
||||
{"key": "game_experts_2g_16k_partial_2s_cap95", **_GAME2, "stt_pace": _PARTIAL_2S, "gpu_cap": GPU_CAP},
|
||||
{"key": "game_experts_2g_16k_finals_only", **_GAME2, "stt_pace": _FINALS_ONLY},
|
||||
]
|
||||
LEVEL_SETS = {"levels": LEVELS, "together": TOGETHER, "live": LIVE, "tune": TUNED}
|
||||
INSTALL_URL = "https://github.com/Niko1221/Strata/releases/latest/download/install.ps1"
|
||||
@@ -298,6 +312,10 @@ class SttLoad:
|
||||
self.cmd += ["--opt", f"{key}={'none' if v is None else v}"]
|
||||
self.proc: subprocess.Popen | None = None
|
||||
|
||||
@property
|
||||
def pid(self) -> int | None:
|
||||
return self.proc.pid if self.proc else None
|
||||
|
||||
def _read(self) -> dict:
|
||||
try:
|
||||
return read_json(self.out)
|
||||
@@ -463,8 +481,12 @@ def measure_levels(r: StageResult, s: Strata, preset: dict, context: int, levels
|
||||
continue
|
||||
sets = list(lv["sets"])
|
||||
if lv.get("stt_room"):
|
||||
where = (row["context"], lv["parallel"], lv["vision"])
|
||||
where = (row["context"], lv["parallel"], lv["vision"], lv.get("other_room_mib", 0))
|
||||
if where not in rooms:
|
||||
if not stt:
|
||||
# The probe needs the STT model in VRAM, as the app has it.
|
||||
r.error(f"level {lv['key']}", "no STT room was measured with the STT loaded")
|
||||
continue
|
||||
rooms[where] = stt_room(r, s, preset, row, lv, base)
|
||||
row["stt_room"] = rooms[where]
|
||||
sets.append(f"expert-vram={row['stt_room']['expert_vram_gib']}")
|
||||
@@ -483,7 +505,18 @@ def measure_levels(r: StageResult, s: Strata, preset: dict, context: int, levels
|
||||
c.chat([{"role": "user", "content": "안녕"}], max_tokens=8) # loading is not answer time
|
||||
if stt:
|
||||
stt.go()
|
||||
row.update(quick_bench(c))
|
||||
cap = lv.get("gpu_cap")
|
||||
pids = cap_pids(s, stt) if cap else []
|
||||
if cap and not pids:
|
||||
r.error(f"level {lv['key']} gpu cap", "found no engine process to pause")
|
||||
with gpucap.UtilSampler() as util, gpucap.DutyCap(pids, cap or 1.0) as duty:
|
||||
row.update(quick_bench(c))
|
||||
row["gpu_util"] = util.summary(cap)
|
||||
if cap:
|
||||
row["gpu_cap"] = duty.summary()
|
||||
if row["gpu_cap"]["failed_processes"]:
|
||||
r.error(f"level {lv['key']} gpu cap",
|
||||
f"{row['gpu_cap']['failed_processes']} process(es) could not be paused")
|
||||
for err in row["errors"]:
|
||||
r.error(f"level {lv['key']}", err)
|
||||
except Exception as e:
|
||||
@@ -540,10 +573,11 @@ def staging_gib(lines: list[str], default: float = 1.10) -> float:
|
||||
|
||||
|
||||
def expert_budget_gib(total_mib: int, used_mib: int, probe_model_mib: int, staging: float, growth_mib: int,
|
||||
margin_mib: int = STT_ROOM_MARGIN_MIB) -> float:
|
||||
"""--expert-vram (GiB, rounded down to 0.1) that leaves the STT's growth and the margin free.
|
||||
The budget first holds the staging, which the start without experts already has."""
|
||||
spare = total_mib - used_mib - probe_model_mib - growth_mib - margin_mib
|
||||
margin_mib: int = STT_ROOM_MARGIN_MIB, other_mib: int = 0) -> float:
|
||||
"""--expert-vram (GiB, rounded down to 0.1) that leaves the STT's growth, the margin and the room
|
||||
for other programs free. The budget first holds the staging, which the start without experts
|
||||
already has."""
|
||||
spare = total_mib - used_mib - probe_model_mib - growth_mib - margin_mib - other_mib
|
||||
if spare <= 0:
|
||||
return 0.0
|
||||
return math.floor((staging * 1024 + spare) / 1024 * 10) / 10
|
||||
@@ -553,7 +587,7 @@ def stt_room(r: StageResult, s: Strata, preset: dict, row: dict, lv: dict, base:
|
||||
"""Start the placement once without experts (the STT model already loaded), see what the LLM
|
||||
holds, stop it, and work out the expert budget that leaves the STT's ten-speaker growth free."""
|
||||
room: dict = {"stt_growth_mib": stt_growth_mib(s.log.parent.parent), "margin_mib": STT_ROOM_MARGIN_MIB,
|
||||
"expert_vram_gib": 0.0}
|
||||
"other_room_mib": lv.get("other_room_mib", 0), "expert_vram_gib": 0.0}
|
||||
say(" STT 자리 계산: 전문가 층 없이 한 번 켜서 LLM 기본 VRAM을 잽니다")
|
||||
before = gpu_memory_mib()
|
||||
launch = s.launch(preset, row["context"], parallel=lv["parallel"], vision=lv["vision"],
|
||||
@@ -571,7 +605,8 @@ def stt_room(r: StageResult, s: Strata, preset: dict, row: dict, lv: dict, base:
|
||||
room.update(probe_model_mib=after[0] - before[0], used_before_mib=before[0], total_mib=before[1],
|
||||
staging_gib=staging_gib(lines))
|
||||
room["expert_vram_gib"] = expert_budget_gib(before[1], before[0], room["probe_model_mib"],
|
||||
room["staging_gib"], room["stt_growth_mib"])
|
||||
room["staging_gib"], room["stt_growth_mib"],
|
||||
other_mib=room["other_room_mib"])
|
||||
if room["expert_vram_gib"] == 0:
|
||||
r.note(f"{lv['key']}: no room for experts next to the STT; started with --expert-vram 0")
|
||||
else:
|
||||
@@ -580,6 +615,16 @@ def stt_room(r: StageResult, s: Strata, preset: dict, row: dict, lv: dict, base:
|
||||
return room
|
||||
|
||||
|
||||
def cap_pids(s: Strata, stt: SttLoad | None) -> list[int]:
|
||||
"""The processes the GPU cap pauses together: the engine and the STT worker, with their children.
|
||||
None when the engine is not found (pausing only the STT would not be the cap)."""
|
||||
procs = gpucap.process_table()
|
||||
roots = gpucap.engine_roots(procs, s.home)
|
||||
if not roots:
|
||||
return []
|
||||
return gpucap.with_children(procs, roots + ([stt.pid] if stt and stt.pid else []))
|
||||
|
||||
|
||||
def ask_for_a_game() -> bool:
|
||||
"""The live stage needs someone to start a game first. False when nobody is there or they say no."""
|
||||
if not interactive():
|
||||
|
||||
Reference in New Issue
Block a user