javis-bot 7792be254a perf(brain): keep ollama models resident to kill voice cold-start latency ...

Default ollama keep_alive is 5 min, so any voice turn after a short idle paid
a full Qwen3 8B reload into the GPU (~30-60s) before replying. Set
OLLAMA_KEEP_ALIVE=-1 so the chat + embed models stay loaded.

Measured: post-idle turn ~60s -> steady-state ~4.5s per turn (model pinned
"Forever" in `ollama ps`).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-06-12 19:12:23 +09:00

4.5 KiB

Raw Blame History

View Raw