perf(brain): keep ollama models resident to kill voice cold-start latency
Default ollama keep_alive is 5 min, so any voice turn after a short idle paid a full Qwen3 8B reload into the GPU (~30-60s) before replying. Set OLLAMA_KEEP_ALIVE=-1 so the chat + embed models stay loaded. Measured: post-idle turn ~60s -> steady-state ~4.5s per turn (model pinned "Forever" in `ollama ps`). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -17,6 +17,11 @@ services:
|
|||||||
ollama:
|
ollama:
|
||||||
image: ollama/ollama:latest
|
image: ollama/ollama:latest
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
# Keep the chat + embed models resident so voice turns never pay a cold
|
||||||
|
# reload. Default keep_alive is 5 min, so every post-idle turn took
|
||||||
|
# ~30-60s while Qwen3 8B reloaded into the GPU. -1 = never unload.
|
||||||
|
OLLAMA_KEEP_ALIVE: "-1"
|
||||||
volumes:
|
volumes:
|
||||||
- ollama_models:/root/.ollama
|
- ollama_models:/root/.ollama
|
||||||
# GPU: needs nvidia-container-toolkit on the host (CDI). Verified on the
|
# GPU: needs nvidia-container-toolkit on the host (CDI). Verified on the
|
||||||
|
|||||||
Reference in New Issue
Block a user