docs: warn against setting OLLAMA_INTENT_MODEL larger than the chat model
Some checks failed
Release / semantic-release (push) Successful in 20s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m19s
Release / build-linux (push) Failing after 7m13s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled

A deployment had OLLAMA_INTENT_MODEL=qwen2.5:7b while the chat model was a 4b,
so every auxiliary call (intent judge, tool router, place extraction, query
decomposition) ran on the bigger, slower model and added latency to each
command. Make the .env.example comment state the invariant explicitly.
This commit is contained in:
javis-bot
2026-06-24 17:57:30 +09:00
parent 7bb9718c34
commit 23b1fe692b

View File

@@ -65,6 +65,9 @@ OLLAMA_CHAT_MODEL=qwen2.5:3b
# default qwen2.5:3b, which ollama-init pulls automatically. Set it equal to
# OLLAMA_CHAT_MODEL to run everything on one resident model instead (saves VRAM
# at the cost of slower routing when the chat model is large).
# NEVER set this LARGER than OLLAMA_CHAT_MODEL: the auxiliary calls would then
# run on the bigger, slower model and add latency to every command (the exact
# opposite of the split's purpose). Keep it <= the chat model, blank, or equal.
OLLAMA_INTENT_MODEL=
OLLAMA_EMBED_MODEL=nomic-embed-text
WHISPER_MODEL=small