perf: cap chat output tokens via ollama_num_predict to bound reply latency

Spoken (TTS) replies are 1-2 sentences, so an unbounded num_predict only
exposes the worst case where the chat model rambles or loops. Add an
ollama_num_predict config (default 512, 0 disables) wired into the reply
loop's chat call on both the native- and text-tool paths. The 512-token
headroom stays well above this app's short tool-call JSON, so capping never
truncates a tool call. This keeps the user's quality model instead of
downgrading it. Configurable in the container via OLLAMA_NUM_PREDICT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-23 15:33:45 +09:00
parent c189ce2e65
commit 5ee47827f3
7 changed files with 146 additions and 4 deletions

View File

@@ -4,6 +4,7 @@
"ollama_base_url": "${OLLAMA_BASE_URL}",
"ollama_embed_model": "${OLLAMA_EMBED_MODEL}",
"ollama_chat_model": "${OLLAMA_CHAT_MODEL}",
"ollama_num_predict": "${OLLAMA_NUM_PREDICT}",
"intent_judge_model": "${OLLAMA_INTENT_MODEL}",
"tts_enabled": true,
"tts_engine": "${TTS_ENGINE}",