perf(brain): keep ollama models resident to kill voice cold-start latency

Default ollama keep_alive is 5 min, so any voice turn after a short idle paid
a full Qwen3 8B reload into the GPU (~30-60s) before replying. Set
OLLAMA_KEEP_ALIVE=-1 so the chat + embed models stay loaded.

Measured: post-idle turn ~60s -> steady-state ~4.5s per turn (model pinned
"Forever" in `ollama ps`).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-12 19:12:23 +09:00
parent b17961e9e3
commit 7792be254a

View File

@@ -17,6 +17,11 @@ services:
ollama:
image: ollama/ollama:latest
restart: unless-stopped
environment:
# Keep the chat + embed models resident so voice turns never pay a cold
# reload. Default keep_alive is 5 min, so every post-idle turn took
# ~30-60s while Qwen3 8B reloaded into the GPU. -1 = never unload.
OLLAMA_KEEP_ALIVE: "-1"
volumes:
- ollama_models:/root/.ollama
# GPU: needs nvidia-container-toolkit on the host (CDI). Verified on the