tkrmagid/javis_bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 20 Wiki Activity

feat(deploy): add brain/app roles for GPU-split deployment

Browse Source 
Run the GPU-heavy compute (Whisper STT, Ollama LLM, TTS) on a dedicated
brain host and the bot + browser on a separate no-GPU app host, talking
over the LAN.

- supervisord: new `brain` role (bridge only) and `app` role (bot +
  desktop/Chrome, no local bridge); `full`/`browser`/`bot` unchanged.
- compose: make BRIDGE_URL overridable (was hardcoded to loopback, which
  prevented the bot from reaching a remote bridge).
- docs: DEPLOY.md "GPU split" layout with per-host .env blocks; document
  the new roles in run-if-role.sh and .env.example.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-07-22 15:56:51 +09:00
parent db51bb92e6
commit 8562548571
5 changed files with 78 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
.env.example
View File

@@ -29,7 +29,9 @@ BRIDGE_PORT=8765
JARVIS_BRAIN_ENABLED=1
JARVIS_TTS_ENABLED=1
# faster-whisper device/compute. GPU by default (RTX 5050 / sm_120, verified).
# Falls back to CPU automatically if no GPU is passed to the container.
# On a NO-GPU host (e.g. an app host that offloads the LLM to a separate GPU box)
# set WHISPER_DEVICE=cpu AND WHISPER_COMPUTE_TYPE=int8 — float16 is GPU-only, so
# CPU needs int8 (STT is slower on CPU).
WHISPER_DEVICE=cuda
WHISPER_COMPUTE_TYPE=float16
# Optional explicit Piper voice model (.onnx). If empty, the jarvis default is used.
@@ -203,6 +205,10 @@ VOICE_SILENCE_MS=800
# full (default) everything in one container
# browser ONLY the desktop + Chrome + control-server (driven over the LAN)
# bot ONLY the bot + bridge + TTS (drives a REMOTE browser)
# brain ONLY the bridge (STT + reply + TTS) on a GPU host — no bot/desktop.
# The app host reaches it via BRIDGE_URL. See docs/DEPLOY.md "GPU split".
# app the bot + desktop/Chrome, but NOT the local bridge: offloads
# STT/LLM/TTS to a remote brain host (set BRIDGE_URL=http://<brain>:8765).
JARVIS_ROLE=full
# --- GPU per OS: pick the matching compose override via COMPOSE_FILE ---

15
docker-compose.yml
View File

@@ -106,10 +106,17 @@ services:
# on short/noisy Discord speech. Lower to 1 for minimum latency.
STT_BEAM_SIZE: ${STT_BEAM_SIZE:-5}
VOICE_SILENCE_MS: ${VOICE_SILENCE_MS:-600}
BRIDGE_URL: http://127.0.0.1:8765
# Split-deployment role: full (default, all-in-one), browser (only the
# desktop + Chrome + CDP, reused over the LAN), or bot (only bot + bridge
# + TTS, driving a remote browser via CDP_HOST). See docker/run-if-role.sh.
# Where the bot reaches the brain bridge (STT + reply + TTS). Loopback for
# all-in-one. In a GPU-split layout the bot host (JARVIS_ROLE=app) points
# this at the brain host: BRIDGE_URL=http://192.168.10.9:8765 (that host
# runs JARVIS_ROLE=brain with BRIDGE_HOST=0.0.0.0 + SETTINGS_PUBLISH_BIND=0.0.0.0).
BRIDGE_URL: ${BRIDGE_URL:-http://127.0.0.1:8765}
# Split-deployment role (see docker/run-if-role.sh):
# full (default) everything in one container
# browser only desktop + Chrome + control-server (driven over the LAN)
# bot only bot + bridge + TTS (drives a remote browser)
# brain only the bridge (STT + reply + TTS) — GPU host, no bot/desktop
# app bot + browser/desktop, NO local bridge (uses a remote brain)
JARVIS_ROLE: ${JARVIS_ROLE:-full}
# Chrome CDP bind address INSIDE the container. 0.0.0.0 lets a remote bot
# (JARVIS_ROLE=bot on another PC) drive this host's browser. Loopback by

4
docker/run-if-role.sh
View File

@@ -11,6 +11,10 @@
# JARVIS_ROLE=browser only the desktop + Chrome + CDP (reused over the LAN)
# JARVIS_ROLE=bot only the bot + bridge + TTS (drives a remote browser
# via CDP_HOST/CDP_PORT)
# JARVIS_ROLE=brain only the bridge (STT + reply + TTS) on a GPU host; no
# bot, no desktop. The app host reaches it via BRIDGE_URL.
# JARVIS_ROLE=app the bot + desktop/Chrome, but NOT the local bridge:
# offloads STT/LLM/TTS to a remote brain host (BRIDGE_URL)
set -e
want="$1"; shift
role="${JARVIS_ROLE:-full}"

16
docker/supervisord.conf
View File

@@ -14,7 +14,7 @@ serverurl=unix:///run/supervisor.sock
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[program:xvnc]
command=/app/docker/run-if-role.sh full,browser /app/docker/run-xvnc.sh
command=/app/docker/run-if-role.sh full,browser,app /app/docker/run-xvnc.sh
priority=100
autorestart=true
stdout_logfile=/dev/stdout
@@ -23,7 +23,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:pulse]
command=/app/docker/run-if-role.sh full,browser /app/docker/run-pulse.sh
command=/app/docker/run-if-role.sh full,browser,app /app/docker/run-pulse.sh
priority=150
autorestart=true
stdout_logfile=/dev/stdout
@@ -32,7 +32,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:xfce]
command=/app/docker/run-if-role.sh full,browser /app/docker/run-xfce.sh
command=/app/docker/run-if-role.sh full,browser,app /app/docker/run-xfce.sh
priority=200
autorestart=true
stdout_logfile=/dev/stdout
@@ -41,7 +41,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:novnc]
command=/app/docker/run-if-role.sh full,browser websockify --web=/usr/share/novnc 6080 localhost:5901
command=/app/docker/run-if-role.sh full,browser,app websockify --web=/usr/share/novnc 6080 localhost:5901
priority=250
autorestart=true
stdout_logfile=/dev/stdout
@@ -53,7 +53,7 @@ stderr_logfile_maxbytes=0
# the bridge via the `edge-tts` package — no warm model/worker is needed.)
[program:bridge]
command=/app/docker/run-if-role.sh full,bot /opt/venv/bin/python -m bridge.server
command=/app/docker/run-if-role.sh full,bot,brain /opt/venv/bin/python -m bridge.server
directory=/app
priority=300
autorestart=true
@@ -63,7 +63,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:chrome]
command=/app/docker/run-if-role.sh full,browser /app/docker/run-chrome.sh
command=/app/docker/run-if-role.sh full,browser,app /app/docker/run-chrome.sh
priority=350
autorestart=true
stdout_logfile=/dev/stdout
@@ -75,7 +75,7 @@ stderr_logfile_maxbytes=0
; Browser-control HTTP endpoint on the BROWSER HOST. A remote `bot` posts
; commands here so xdotool / CDP run on THIS machine (real input on this
; screen). Only meaningful in full/browser roles. Internal network only.
command=/app/docker/run-if-role.sh full,browser node /app/bot/scripts/stream-test/control-server.mjs
command=/app/docker/run-if-role.sh full,browser,app node /app/bot/scripts/stream-test/control-server.mjs
directory=/app/bot
priority=360
autorestart=true
@@ -85,7 +85,7 @@ stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:bot]
command=/app/docker/run-if-role.sh full,bot /app/docker/run-bot.sh
command=/app/docker/run-if-role.sh full,bot,app /app/docker/run-bot.sh
directory=/app/bot
priority=400
autorestart=true

48
docs/DEPLOY.md
View File

@@ -79,6 +79,54 @@ On the LLM host, make Ollama listen on the LAN (`OLLAMA_HOST=0.0.0.0:11434`) and
pull the models there (`ollama pull <chat>`, `<intent>`, `<embed>`). The compose
respects `OLLAMA_BASE_URL`, defaulting to the in-stack `ollama` service when unset.
## C. GPU split: brain host (.9, GPU) + app host (.5, no GPU)
All the GPU-heavy compute (Whisper STT, the Ollama LLM, and TTS) runs on the
**brain host** (a GPU box, e.g. `192.168.10.9`). The **app host** (no GPU, e.g.
`192.168.10.5`) runs the Discord bot and the on-screen browser, and offloads
every STT/LLM/TTS call to the brain over the LAN.
Flow: the bot (app host) POSTs captured voice to the brain's bridge; the brain
runs STT -> LLM -> TTS on the GPU and returns the reply audio; when the LLM
drives the browser, the brain's `controlBrowser` tool POSTs to the app host's
control-server (port 8777), so Chrome moves on the app host's screen.
### Brain host (.9 — GPU: STT + LLM + TTS)
```
# .env
JARVIS_ROLE=brain
BRIDGE_HOST=0.0.0.0 # bridge listens on the LAN
SETTINGS_PUBLISH_BIND=0.0.0.0 # publish bridge port 8765 to the LAN
OLLAMA_BASE_URL=http://192.168.10.9:11434 # Ollama on this host (bind 0.0.0.0)
OLLAMA_CHAT_MODEL=qwen2.5:7b
WHISPER_DEVICE=cuda
WHISPER_COMPUTE_TYPE=float16
BROWSER_CONTROL_URL=http://192.168.10.5:8777 # drive the app host's Chrome
COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml # GPU (":" on Linux)
# Ollama runs on this host (systemd or a container) listening on 0.0.0.0:11434.
docker compose up -d javis # brain bridge only (no bot, no desktop)
```
### App host (.5 — bot + browser, no GPU)
```
# .env
JARVIS_ROLE=app
BRIDGE_URL=http://192.168.10.9:8765 # offload STT/LLM/TTS to the brain
CDP_PUBLISH_BIND=0.0.0.0 # publish control-server 8777 to the LAN
STREAM_HW=0 # no NVENC on a no-GPU host
DISCORD_SELFBOT_TOKEN=...
DISCORD_GUILD_ID=...
# no GPU → leave COMPOSE_FILE unset (base compose only)
docker compose up -d javis # bot + desktop + Chrome + control-server
```
The `brain` and `app` roles are backward-compatible additions to `full`/`browser`
/`bot`; see `docker/run-if-role.sh`.
## Windows 11 notes
- Install the NVIDIA driver on Windows and enable GPU in Docker Desktop