feat(compose): allow external Ollama via OLLAMA_BASE_URL override
For the split where the LLM runs on a separate LAN host (e.g. a GPU box at 192.168.10.9) and the app stack runs elsewhere (192.168.10.5), the brain must reach Ollama over the network. The javis service hardcoded OLLAMA_BASE_URL=http://ollama:11434, ignoring any .env value, so the app could only use the in-stack Ollama. Make it ${OLLAMA_BASE_URL:-http://ollama:11434}: default unchanged (all-in-one), overridable to point at an external Ollama. Document the external-Ollama setup in .env.example and DEPLOY.md, refresh the browser-host IP examples (.9→.5), and de-hardcode the .9 example in novnc.ts.
This commit is contained in:
@@ -51,7 +51,13 @@ MELO_FALLBACK_PIPER=0
|
|||||||
# Jarvis brain (Ollama-backed). In Docker these populate the rendered
|
# Jarvis brain (Ollama-backed). In Docker these populate the rendered
|
||||||
# config (docker/jarvis-config.template.json). See src/jarvis/config.py.
|
# config (docker/jarvis-config.template.json). See src/jarvis/config.py.
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# In docker-compose this is overridden to http://ollama:11434 automatically.
|
# In docker-compose this defaults to the in-stack `ollama` service
|
||||||
|
# (http://ollama:11434). To use an EXTERNAL Ollama on another LAN machine (e.g. a
|
||||||
|
# GPU host), set it here to that host's IP — the compose now respects this value:
|
||||||
|
# OLLAMA_BASE_URL=http://192.168.10.9:11434 # Ollama runs on the .9 host
|
||||||
|
# In that split setup: do NOT start the in-stack ollama/ollama-init (run
|
||||||
|
# `docker compose up -d javis`), pull the models on the LLM host, and make that
|
||||||
|
# host's Ollama listen on the LAN (OLLAMA_HOST=0.0.0.0:11434).
|
||||||
OLLAMA_BASE_URL=http://127.0.0.1:11434
|
OLLAMA_BASE_URL=http://127.0.0.1:11434
|
||||||
# qwen2.5:3b — small non-reasoning instruct model. ~2.4GB, runs 100% on the GPU
|
# qwen2.5:3b — small non-reasoning instruct model. ~2.4GB, runs 100% on the GPU
|
||||||
# (the 8B offloads ~8% to CPU), warm voice turns ~2-4s vs ~5-7s on 8B. Clean
|
# (the 8B offloads ~8% to CPU), warm voice turns ~2-4s vs ~5-7s on 8B. Clean
|
||||||
|
|||||||
@@ -3,7 +3,8 @@
|
|||||||
*
|
*
|
||||||
* Does not broadcast natively into Discord. Instead it shares a noVNC web URL
|
* Does not broadcast natively into Discord. Instead it shares a noVNC web URL
|
||||||
* that anyone can open in a browser to watch (and optionally control) the VNC
|
* that anyone can open in a browser to watch (and optionally control) the VNC
|
||||||
* desktop live. Set NOVNC_URL in .env (e.g. http://192.168.10.9:6080/vnc.html).
|
* desktop live. Set NOVNC_URL in .env to the browser host's LAN IP
|
||||||
|
* (e.g. http://<browser-host-ip>:6080/vnc.html).
|
||||||
*
|
*
|
||||||
* Stand up noVNC once on the host with websockify, e.g.:
|
* Stand up noVNC once on the host with websockify, e.g.:
|
||||||
* websockify --web=/usr/share/novnc 6080 localhost:5901
|
* websockify --web=/usr/share/novnc 6080 localhost:5901
|
||||||
@@ -22,7 +23,7 @@ export class NoVncStreamer implements ScreenStreamer {
|
|||||||
|
|
||||||
async start(_ctx: StreamContext): Promise<string> {
|
async start(_ctx: StreamContext): Promise<string> {
|
||||||
if (!this.config.novncUrl) {
|
if (!this.config.novncUrl) {
|
||||||
return "NOVNC_URL이 설정되지 않았습니다 (.env). 예: http://192.168.10.9:6080/vnc.html";
|
return "NOVNC_URL이 설정되지 않았습니다 (.env). 예: http://<브라우저호스트IP>:6080/vnc.html";
|
||||||
}
|
}
|
||||||
this.active = true;
|
this.active = true;
|
||||||
return `🖥️ VNC 화면 실시간 보기 (브라우저): ${this.config.novncUrl}`;
|
return `🖥️ VNC 화면 실시간 보기 (브라우저): ${this.config.novncUrl}`;
|
||||||
|
|||||||
@@ -66,8 +66,13 @@ services:
|
|||||||
- path: .env
|
- path: .env
|
||||||
required: false
|
required: false
|
||||||
environment:
|
environment:
|
||||||
# Point the brain at the ollama service and the bot at the in-container bridge.
|
# Point the brain at Ollama. Default is the in-stack `ollama` service
|
||||||
OLLAMA_BASE_URL: http://ollama:11434
|
# (all-in-one). Override OLLAMA_BASE_URL in .env to reach an EXTERNAL Ollama
|
||||||
|
# over the LAN (e.g. a GPU host at http://192.168.10.9:11434) — used when
|
||||||
|
# this stack runs on a separate machine from the LLM. When overridden, do
|
||||||
|
# NOT start the in-stack `ollama`/`ollama-init` services (bring the app up
|
||||||
|
# with `docker compose up -d javis`) and pull the models on the LLM host.
|
||||||
|
OLLAMA_BASE_URL: ${OLLAMA_BASE_URL:-http://ollama:11434}
|
||||||
OLLAMA_CHAT_MODEL: ${OLLAMA_CHAT_MODEL:-qwen2.5:3b}
|
OLLAMA_CHAT_MODEL: ${OLLAMA_CHAT_MODEL:-qwen2.5:3b}
|
||||||
# Auxiliary small-model calls (intent judge, tool router, arg extraction,
|
# Auxiliary small-model calls (intent judge, tool router, arg extraction,
|
||||||
# query decomposition) run on this fast model so the big chat model only
|
# query decomposition) run on this fast model so the big chat model only
|
||||||
@@ -112,7 +117,7 @@ services:
|
|||||||
CDP_BIND: ${CDP_BIND:-127.0.0.1}
|
CDP_BIND: ${CDP_BIND:-127.0.0.1}
|
||||||
CDP_PORT: ${CDP_PORT:-9222}
|
CDP_PORT: ${CDP_PORT:-9222}
|
||||||
# Where the bot drives Chrome. Loopback for full/browser; on a remote bot
|
# Where the bot drives Chrome. Loopback for full/browser; on a remote bot
|
||||||
# set CDP_HOST to the browser host's LAN IP (e.g. 192.168.10.9).
|
# set CDP_HOST to the browser host's LAN IP (e.g. 192.168.10.5).
|
||||||
CDP_HOST: ${CDP_HOST:-127.0.0.1}
|
CDP_HOST: ${CDP_HOST:-127.0.0.1}
|
||||||
# Browser-control endpoint. The browser host serves it (BIND/PORT); a
|
# Browser-control endpoint. The browser host serves it (BIND/PORT); a
|
||||||
# remote bot sets BROWSER_CONTROL_URL=http://<browser-host>:8777 so its
|
# remote bot sets BROWSER_CONTROL_URL=http://<browser-host>:8777 so its
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ The on-screen Chrome, real mouse/keyboard (xdotool) and screen live on the
|
|||||||
**browser host**. Your PC runs the **bot** and drives that browser over the
|
**browser host**. Your PC runs the **bot** and drives that browser over the
|
||||||
internal network — no auth (internal only).
|
internal network — no auth (internal only).
|
||||||
|
|
||||||
### Browser host (the LAN machine that shows Chrome, e.g. 192.168.10.9)
|
### Browser host (the LAN machine that shows Chrome, e.g. 192.168.10.5)
|
||||||
|
|
||||||
```
|
```
|
||||||
# .env
|
# .env
|
||||||
@@ -50,7 +50,7 @@ Watch it on this machine’s VNC (`localhost:5901`) / noVNC (`localhost:6080`).
|
|||||||
```
|
```
|
||||||
# .env
|
# .env
|
||||||
JARVIS_ROLE=bot
|
JARVIS_ROLE=bot
|
||||||
BROWSER_CONTROL_URL=http://192.168.10.9:8777 # the browser host's LAN IP
|
BROWSER_CONTROL_URL=http://192.168.10.5:8777 # the browser host's LAN IP
|
||||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu/macOS (":" )
|
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # Ubuntu/macOS (":" )
|
||||||
# COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml # Windows 11 (";" )
|
# COMPOSE_FILE=docker-compose.yml;docker-compose.gpu-windows.yml # Windows 11 (";" )
|
||||||
DISCORD_SELFBOT_TOKEN=...
|
DISCORD_SELFBOT_TOKEN=...
|
||||||
@@ -63,6 +63,22 @@ The bot’s `controlBrowser` tool posts commands to `BROWSER_CONTROL_URL`, so
|
|||||||
"네이버에서 X 검색", "구글로 돌아가" etc. drive the **browser host’s** Chrome with real
|
"네이버에서 X 검색", "구글로 돌아가" etc. drive the **browser host’s** Chrome with real
|
||||||
human-style input (visible on its VNC).
|
human-style input (visible on its VNC).
|
||||||
|
|
||||||
|
### External Ollama (LLM on a separate host)
|
||||||
|
|
||||||
|
To run Ollama on a different LAN machine (e.g. a dedicated GPU host at
|
||||||
|
`192.168.10.9`) instead of inside this stack:
|
||||||
|
|
||||||
|
```
|
||||||
|
# .env on the app host
|
||||||
|
OLLAMA_BASE_URL=http://192.168.10.9:11434
|
||||||
|
```
|
||||||
|
|
||||||
|
Then start the app WITHOUT the in-stack LLM services: `docker compose up -d javis`
|
||||||
|
(do not `docker compose up -d`, which would also start `ollama`/`ollama-init`).
|
||||||
|
On the LLM host, make Ollama listen on the LAN (`OLLAMA_HOST=0.0.0.0:11434`) and
|
||||||
|
pull the models there (`ollama pull <chat>`, `<intent>`, `<embed>`). The compose
|
||||||
|
respects `OLLAMA_BASE_URL`, defaulting to the in-stack `ollama` service when unset.
|
||||||
|
|
||||||
## Windows 11 notes
|
## Windows 11 notes
|
||||||
|
|
||||||
- Install the NVIDIA driver on Windows and enable GPU in Docker Desktop
|
- Install the NVIDIA driver on Windows and enable GPU in Docker Desktop
|
||||||
|
|||||||
Reference in New Issue
Block a user