perf(brain): keep ollama models resident to kill voice cold-start latency
Default ollama keep_alive is 5 min, so any voice turn after a short idle paid a full Qwen3 8B reload into the GPU (~30-60s) before replying. Set OLLAMA_KEEP_ALIVE=-1 so the chat + embed models stay loaded. Measured: post-idle turn ~60s -> steady-state ~4.5s per turn (model pinned "Forever" in `ollama ps`). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -17,6 +17,11 @@ services:
|
||||
ollama:
|
||||
image: ollama/ollama:latest
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
# Keep the chat + embed models resident so voice turns never pay a cold
|
||||
# reload. Default keep_alive is 5 min, so every post-idle turn took
|
||||
# ~30-60s while Qwen3 8B reloaded into the GPU. -1 = never unload.
|
||||
OLLAMA_KEEP_ALIVE: "-1"
|
||||
volumes:
|
||||
- ollama_models:/root/.ollama
|
||||
# GPU: needs nvidia-container-toolkit on the host (CDI). Verified on the
|
||||
|
||||
Reference in New Issue
Block a user