feat(deploy): add brain/app roles for GPU-split deployment
Run the GPU-heavy compute (Whisper STT, Ollama LLM, TTS) on a dedicated brain host and the bot + browser on a separate no-GPU app host, talking over the LAN. - supervisord: new `brain` role (bridge only) and `app` role (bot + desktop/Chrome, no local bridge); `full`/`browser`/`bot` unchanged. - compose: make BRIDGE_URL overridable (was hardcoded to loopback, which prevented the bot from reaching a remote bridge). - docs: DEPLOY.md "GPU split" layout with per-host .env blocks; document the new roles in run-if-role.sh and .env.example. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -79,6 +79,54 @@ On the LLM host, make Ollama listen on the LAN (`OLLAMA_HOST=0.0.0.0:11434`) and
|
||||
pull the models there (`ollama pull <chat>`, `<intent>`, `<embed>`). The compose
|
||||
respects `OLLAMA_BASE_URL`, defaulting to the in-stack `ollama` service when unset.
|
||||
|
||||
## C. GPU split: brain host (.9, GPU) + app host (.5, no GPU)
|
||||
|
||||
All the GPU-heavy compute (Whisper STT, the Ollama LLM, and TTS) runs on the
|
||||
**brain host** (a GPU box, e.g. `192.168.10.9`). The **app host** (no GPU, e.g.
|
||||
`192.168.10.5`) runs the Discord bot and the on-screen browser, and offloads
|
||||
every STT/LLM/TTS call to the brain over the LAN.
|
||||
|
||||
Flow: the bot (app host) POSTs captured voice to the brain's bridge; the brain
|
||||
runs STT -> LLM -> TTS on the GPU and returns the reply audio; when the LLM
|
||||
drives the browser, the brain's `controlBrowser` tool POSTs to the app host's
|
||||
control-server (port 8777), so Chrome moves on the app host's screen.
|
||||
|
||||
### Brain host (.9 — GPU: STT + LLM + TTS)
|
||||
|
||||
```
|
||||
# .env
|
||||
JARVIS_ROLE=brain
|
||||
BRIDGE_HOST=0.0.0.0 # bridge listens on the LAN
|
||||
SETTINGS_PUBLISH_BIND=0.0.0.0 # publish bridge port 8765 to the LAN
|
||||
OLLAMA_BASE_URL=http://192.168.10.9:11434 # Ollama on this host (bind 0.0.0.0)
|
||||
OLLAMA_CHAT_MODEL=qwen2.5:7b
|
||||
WHISPER_DEVICE=cuda
|
||||
WHISPER_COMPUTE_TYPE=float16
|
||||
BROWSER_CONTROL_URL=http://192.168.10.5:8777 # drive the app host's Chrome
|
||||
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # GPU (":" on Linux)
|
||||
|
||||
# Ollama runs on this host (systemd or a container) listening on 0.0.0.0:11434.
|
||||
docker compose up -d javis # brain bridge only (no bot, no desktop)
|
||||
```
|
||||
|
||||
### App host (.5 — bot + browser, no GPU)
|
||||
|
||||
```
|
||||
# .env
|
||||
JARVIS_ROLE=app
|
||||
BRIDGE_URL=http://192.168.10.9:8765 # offload STT/LLM/TTS to the brain
|
||||
CDP_PUBLISH_BIND=0.0.0.0 # publish control-server 8777 to the LAN
|
||||
STREAM_HW=0 # no NVENC on a no-GPU host
|
||||
DISCORD_SELFBOT_TOKEN=...
|
||||
DISCORD_GUILD_ID=...
|
||||
# no GPU → leave COMPOSE_FILE unset (base compose only)
|
||||
|
||||
docker compose up -d javis # bot + desktop + Chrome + control-server
|
||||
```
|
||||
|
||||
The `brain` and `app` roles are backward-compatible additions to `full`/`browser`
|
||||
/`bot`; see `docker/run-if-role.sh`.
|
||||
|
||||
## Windows 11 notes
|
||||
|
||||
- Install the NVIDIA driver on Windows and enable GPU in Docker Desktop
|
||||
|
||||
Reference in New Issue
Block a user