diff --git a/README.md b/README.md index 22b961d..697b615 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,7 @@ COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml > Linux와 Windows는 GPU를 컨테이너에 넣는 방식이 달라서 override 파일이 갈립니다. Linux는 CDI(`devices: nvidia.com/gpu=all`), Windows(Docker Desktop)는 Compose의 `deploy.resources.reservations.devices`(`driver: nvidia`)를 씁니다. 호스트 사전 준비는 아래 "GPU 가속" 절 참고. `docker compose up` 한 번이면 자동으로: -- Ollama 서버가 뜨고, `ollama-init`이 채팅/임베딩 모델을 **자동 pull** +- Ollama 서버가 뜨고, `ollama-init`이 채팅/보조(의도·라우팅)/임베딩 모델을 **자동 pull** (보조 모델 `OLLAMA_INTENT_MODEL`은 기본 `qwen2.5:3b`로, 큰 채팅 모델은 답변에만 쓰고 내부 분류 호출은 이 작은 모델이 처리) - VNC+XFCE 데스크톱 + 크롬 + Python 브릿지가 기동 - Whisper STT 모델 / Piper TTS 음성 자동 다운로드(볼륨에 캐시) diff --git a/docs/llm_contexts.md b/docs/llm_contexts.md index f16ab45..336cae0 100644 --- a/docs/llm_contexts.md +++ b/docs/llm_contexts.md @@ -26,7 +26,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i - **File**: [src/jarvis/listening/intent_judge.py](src/jarvis/listening/intent_judge.py) — `IntentJudge.evaluate()`. - **Trigger**: on a speech segment *only if* there is an engagement signal (wake word detected, hot-window active, or TTS playing). Pure ambient speech skips it. -- **Model / gating**: `cfg.intent_judge_model` (default `gemma4:e2b`, ~2B). Falls back to text-based wake detection if Ollama is unavailable. +- **Model / gating**: `cfg.intent_judge_model`. Code-level default `gemma4:e2b` (~2B); the **Docker stack** renders it from `OLLAMA_INTENT_MODEL` (default `qwen2.5:3b`, pulled by `ollama-init`), kept deliberately **separate from `ollama_chat_model`** so this judge and the tool router (#3, #7) run on a small fast model while the big chat model is reserved for the spoken answer. Setting `OLLAMA_INTENT_MODEL = OLLAMA_CHAT_MODEL` folds them back onto one resident model. Falls back to text-based wake detection if Ollama is unavailable. - **Inputs**: - Rolling transcript buffer (last 120s, with timestamps) - Wake-word timestamp (if any), normalised aliases @@ -246,7 +246,7 @@ user input 3. Pre-warm the intent-judge model before TTS finishes. 4. Cache tool-router (#7) output by query hash. 5. Give each digest its own timeout budget rather than sharing `llm_digest_timeout_sec` (today a slow memory digest can starve the max-turn digest). -6. Consider single-model deployments: router+planner prefer `intent_judge_model`; loading a second model hurts cold-start latency on small hardware. +6. Two-model vs single-model tradeoff: the Docker default keeps a **separate** small `intent_judge_model` (`OLLAMA_INTENT_MODEL=qwen2.5:3b`) so routing/judging/extraction don't pay the big chat model's per-call cost — the main win once the GPU holds both models resident. On VRAM-constrained hardware, fold them onto one model by setting `OLLAMA_INTENT_MODEL = OLLAMA_CHAT_MODEL` (saves a resident model at the cost of slower routing when the chat model is large). 7. Narrow `llm_thinking_enabled` to router/planner only, not every context. 8. Reduce `intent_judge_timeout_sec` (15s) or race it against text-based wake detection to avoid blocking the audio loop.