From 23b1fe692b27d9294f630a3abace494ead0155ed Mon Sep 17 00:00:00 2001 From: javis-bot Date: Wed, 24 Jun 2026 17:57:30 +0900 Subject: [PATCH] docs: warn against setting OLLAMA_INTENT_MODEL larger than the chat model A deployment had OLLAMA_INTENT_MODEL=qwen2.5:7b while the chat model was a 4b, so every auxiliary call (intent judge, tool router, place extraction, query decomposition) ran on the bigger, slower model and added latency to each command. Make the .env.example comment state the invariant explicitly. --- .env.example | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.env.example b/.env.example index 05a275a..32ff360 100644 --- a/.env.example +++ b/.env.example @@ -65,6 +65,9 @@ OLLAMA_CHAT_MODEL=qwen2.5:3b # default qwen2.5:3b, which ollama-init pulls automatically. Set it equal to # OLLAMA_CHAT_MODEL to run everything on one resident model instead (saves VRAM # at the cost of slower routing when the chat model is large). +# NEVER set this LARGER than OLLAMA_CHAT_MODEL: the auxiliary calls would then +# run on the bigger, slower model and add latency to every command (the exact +# opposite of the split's purpose). Keep it <= the chat model, blank, or equal. OLLAMA_INTENT_MODEL= OLLAMA_EMBED_MODEL=nomic-embed-text WHISPER_MODEL=small