7da2fcb5e5
Whisper was decoding with beam_size=1 (greedy), the least accurate setting, which hurt recognition on short/accented/noisy Discord-mic speech. Switch the default to beam search (5, Whisper's own default) and stop conditioning on the previous clip's transcript (which causes repetition/drift on isolated short utterances rather than helping). Both are env-tunable (STT_BEAM_SIZE, STT_CONDITION_ON_PREV) so accuracy/latency can be traded without a code change; wired into docker-compose and documented in .env.example.
191 lines
11 KiB
YAML
191 lines
11 KiB
YAML
# ============================================================================
|
|
# Javis Bot — Docker Compose
|
|
# ollama : the LLM backend for the jarvis brain
|
|
# ollama-init : one-shot, auto-pulls the chat + embed models on startup
|
|
# javis : all-in-one container (VNC desktop + Chrome + bridge + bot)
|
|
#
|
|
# Just bring it up — everything (incl. Ollama models) comes up automatically:
|
|
# docker compose up -d --build
|
|
#
|
|
# The Discord credential can be added LAST: without it the desktop, brain
|
|
# bridge, Ollama and models all run; only the bot waits. This deployment runs
|
|
# in userbot mode, so put DISCORD_SELFBOT_TOKEN in .env and re-run
|
|
# `docker compose up -d`. (A normal-bot DISCORD_BOT_TOKEN is optional and only
|
|
# needed for the legacy slash-command bot; leave it blank for userbot mode.)
|
|
#
|
|
# Watch the desktop: VNC viewer -> localhost:5901 (or browser -> localhost:6080)
|
|
# ============================================================================
|
|
services:
|
|
ollama:
|
|
image: ollama/ollama:latest
|
|
restart: unless-stopped
|
|
# Model residency is controlled per-request, not globally. The brain pins
|
|
# the chat model with keep_alive=30m (src/jarvis/llm.py) so voice turns
|
|
# never pay a cold reload, while embeddings pass keep_alive=0
|
|
# (src/jarvis/memory/embeddings.py) so nomic-embed unloads right after use.
|
|
# A global OLLAMA_KEEP_ALIVE=-1 was removed because it also kept the embed
|
|
# model resident forever, wasting VRAM next to the chat model.
|
|
volumes:
|
|
- ollama_models:/root/.ollama
|
|
# GPU is added by a platform override (see docker-compose.gpu-linux.yml /
|
|
# docker-compose.gpu-windows.yml + COMPOSE_FILE in .env). Base stays
|
|
# GPU-agnostic so the same files run on Ubuntu (CDI) and Windows (Desktop).
|
|
|
|
# Auto-pull the models the brain needs, then exit. Idempotent (re-runnable).
|
|
ollama-init:
|
|
image: ollama/ollama:latest
|
|
depends_on:
|
|
- ollama
|
|
restart: "no"
|
|
environment:
|
|
OLLAMA_HOST: http://ollama:11434
|
|
CHAT_MODEL: ${OLLAMA_CHAT_MODEL:-qwen2.5:3b}
|
|
# Small auxiliary model for intent/router/extraction calls (see javis
|
|
# service). Pulled here so the split is ready out of the box.
|
|
INTENT_MODEL: ${OLLAMA_INTENT_MODEL:-qwen2.5:3b}
|
|
EMBED_MODEL: ${OLLAMA_EMBED_MODEL:-nomic-embed-text}
|
|
entrypoint: ["/bin/sh", "-c"]
|
|
command:
|
|
- |
|
|
echo "[ollama-init] waiting for ollama server...";
|
|
until ollama list >/dev/null 2>&1; do sleep 2; done;
|
|
echo "[ollama-init] pulling $$CHAT_MODEL";
|
|
ollama pull "$$CHAT_MODEL";
|
|
if [ -n "$$INTENT_MODEL" ] && [ "$$INTENT_MODEL" != "$$CHAT_MODEL" ]; then
|
|
echo "[ollama-init] pulling $$INTENT_MODEL (auxiliary intent/router model)";
|
|
ollama pull "$$INTENT_MODEL";
|
|
fi;
|
|
echo "[ollama-init] pulling $$EMBED_MODEL";
|
|
ollama pull "$$EMBED_MODEL";
|
|
echo "[ollama-init] models ready.";
|
|
|
|
javis:
|
|
build: .
|
|
restart: unless-stopped
|
|
env_file:
|
|
- path: .env
|
|
required: false
|
|
environment:
|
|
# Point the brain at the ollama service and the bot at the in-container bridge.
|
|
OLLAMA_BASE_URL: http://ollama:11434
|
|
OLLAMA_CHAT_MODEL: ${OLLAMA_CHAT_MODEL:-qwen2.5:3b}
|
|
# Auxiliary small-model calls (intent judge, tool router, arg extraction,
|
|
# query decomposition) run on this fast model so the big chat model only
|
|
# runs for the actual spoken answer. With the GPU on, this is the main
|
|
# per-turn latency win: a command no longer pays the big model's cost 2-3
|
|
# times for routing/extraction. Defaults to qwen2.5:3b (the project's
|
|
# reference small model, clean Korean on classification); set it equal to
|
|
# OLLAMA_CHAT_MODEL to fold everything back onto one resident model.
|
|
OLLAMA_INTENT_MODEL: ${OLLAMA_INTENT_MODEL:-qwen2.5:3b}
|
|
OLLAMA_EMBED_MODEL: ${OLLAMA_EMBED_MODEL:-nomic-embed-text}
|
|
WHISPER_MODEL: ${WHISPER_MODEL:-medium}
|
|
WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
|
|
WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
|
|
# TTS engine. Rendered into /app/config/jarvis.json via envsubst (the
|
|
# bridge reads that JSON BEFORE the env, so it must carry the real engine,
|
|
# not a hardcoded one — otherwise Korean text is read by the English Piper
|
|
# voice). Default edge; .env can override (e.g. piper for offline).
|
|
TTS_ENGINE: ${TTS_ENGINE:-edge}
|
|
# Edge TTS voice + rate (the chosen natural Korean voice). NOTE: edge is an
|
|
# ONLINE engine — reply text is sent to Microsoft and needs internet.
|
|
EDGE_TTS_VOICE: ${EDGE_TTS_VOICE:-ko-KR-HyunsuMultilingualNeural}
|
|
EDGE_TTS_RATE: ${EDGE_TTS_RATE:-+45%}
|
|
# Optional single-language lock for replies (empty = user's own language).
|
|
OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-ko}
|
|
# Drop the pre-loop planner LLM call to cut voice-reply latency on small
|
|
# hardware (the planner adds a full model round-trip per turn).
|
|
PLANNER_ENABLED: ${PLANNER_ENABLED:-0}
|
|
# Lock STT to Korean (skip Whisper auto-detect).
|
|
STT_LANGUAGE: ${STT_LANGUAGE:-ko}
|
|
# Whisper decode accuracy: beam search (5) over greedy (1) lifts recognition
|
|
# on short/noisy Discord speech. Lower to 1 for minimum latency.
|
|
STT_BEAM_SIZE: ${STT_BEAM_SIZE:-5}
|
|
VOICE_SILENCE_MS: ${VOICE_SILENCE_MS:-600}
|
|
BRIDGE_URL: http://127.0.0.1:8765
|
|
# Split-deployment role: full (default, all-in-one), browser (only the
|
|
# desktop + Chrome + CDP, reused over the LAN), or bot (only bot + bridge
|
|
# + TTS, driving a remote browser via CDP_HOST). See docker/run-if-role.sh.
|
|
JARVIS_ROLE: ${JARVIS_ROLE:-full}
|
|
# Chrome CDP bind address INSIDE the container. 0.0.0.0 lets a remote bot
|
|
# (JARVIS_ROLE=bot on another PC) drive this host's browser. Loopback by
|
|
# default so the all-in-one layout stays unreachable off-host.
|
|
CDP_BIND: ${CDP_BIND:-127.0.0.1}
|
|
CDP_PORT: ${CDP_PORT:-9222}
|
|
# Where the bot drives Chrome. Loopback for full/browser; on a remote bot
|
|
# set CDP_HOST to the browser host's LAN IP (e.g. 192.168.10.9).
|
|
CDP_HOST: ${CDP_HOST:-127.0.0.1}
|
|
# Browser-control endpoint. The browser host serves it (BIND/PORT); a
|
|
# remote bot sets BROWSER_CONTROL_URL=http://<browser-host>:8777 so its
|
|
# controlBrowser tool posts there instead of running node locally. Empty
|
|
# on full/browser → the tool runs chrome-control.mjs locally.
|
|
BROWSER_CONTROL_BIND: ${BROWSER_CONTROL_BIND:-0.0.0.0}
|
|
BROWSER_CONTROL_PORT: ${BROWSER_CONTROL_PORT:-8777}
|
|
BROWSER_CONTROL_URL: ${BROWSER_CONTROL_URL:-}
|
|
# Folder of operator *.md instruction files appended to the main reply
|
|
# LLM's system prompt. Bind-mounted from ./agents below; override only to
|
|
# relocate it inside the container.
|
|
AGENTS_DIR: ${AGENTS_DIR:-/app/agents}
|
|
# No hard depends_on ollama: a browser-host (`docker compose up -d javis`)
|
|
# must NOT pull in Ollama. Full/bot layouts start it with a plain
|
|
# `docker compose up -d` (all services); the bridge tolerates Ollama warming
|
|
# up lazily, so start order doesn't matter.
|
|
# GPU is added by a platform override (docker-compose.gpu-linux.yml /
|
|
# docker-compose.gpu-windows.yml). The browser-only host needs no GPU.
|
|
shm_size: "1gb" # Chrome needs a larger /dev/shm
|
|
ports:
|
|
# All published to loopback only by default — VNC/noVNC use a weak default
|
|
# password and the bridge is an unauthenticated internal API, so none
|
|
# should be reachable off-host. For remote access use an SSH tunnel, or
|
|
# set a strong VNC_PASSWORD and override the bind (e.g. VNC_BIND=0.0.0.0).
|
|
# Host VNC port is overridable; this server already runs Xvnc on 5901 so
|
|
# .env pins VNC_PORT=5902.
|
|
- "${VNC_BIND:-127.0.0.1}:${VNC_PORT:-5901}:5901" # VNC
|
|
- "${VNC_BIND:-127.0.0.1}:${NOVNC_PORT:-6080}:6080" # noVNC (browser)
|
|
# Chrome CDP for a remote bot (JARVIS_ROLE=bot). Loopback by default; for a
|
|
# LAN browser-host set CDP_PUBLISH_BIND=0.0.0.0 (internal network, no auth).
|
|
- "${CDP_PUBLISH_BIND:-127.0.0.1}:${CDP_PORT:-9222}:9222" # Chrome CDP
|
|
# Browser-control endpoint a remote bot posts to (real xdotool input runs
|
|
# on THIS host). Published on the LAN for the browser-host layout.
|
|
- "${CDP_PUBLISH_BIND:-127.0.0.1}:${BROWSER_CONTROL_PORT:-8777}:8777" # control-server
|
|
# Settings UI + brain API (bridge). Reach it at http://localhost:8765/settings
|
|
# on the bot host. Requires BRIDGE_HOST=0.0.0.0 (set in .env) to forward.
|
|
- "${SETTINGS_PUBLISH_BIND:-127.0.0.1}:${BRIDGE_PORT:-8765}:8765" # bridge / settings
|
|
# The brain bridge is NOT published: it binds the container's loopback
|
|
# (BRIDGE_HOST=127.0.0.1) and is only consumed by the bot in this same
|
|
# container, so it needs no host port and stays unreachable off-container.
|
|
volumes:
|
|
- javis_data:/data # jarvis db + memory
|
|
- whisper_cache:/root/.cache/huggingface # cached Whisper models
|
|
- piper_voices:/opt/piper-voices # TTS voices
|
|
# Gemini account login for GEMINI_AUTH=oauth real-time search. Bind-mounts a
|
|
# PROJECT-LOCAL dir (./docker/gemini-oauth) into the CLI's ~/.gemini. A
|
|
# project-relative path is used on purpose: it resolves identically on Linux
|
|
# and on Windows Docker Desktop, unlike ${HOME} which is frequently unset
|
|
# when compose is invoked outside a WSL shell (PowerShell/cmd), silently
|
|
# mounting the wrong dir. The mount is writable so the CLI refreshes its
|
|
# token in place.
|
|
#
|
|
# Seed it ONCE from a machine that has a browser + the logged-in Gemini CLI
|
|
# (`npm i -g @google/gemini-cli`, then `gemini` -> "Sign in with Google"):
|
|
# cp -r ~/.gemini/. docker/gemini-oauth/ # Linux / WSL
|
|
# `oauth_creds.json` is the essential credential (holds the refresh token);
|
|
# with GOOGLE_GENAI_USE_GCA=true the CLI forces OAuth, so that one file is
|
|
# what the readiness check + entrypoint warning verify. Copying the WHOLE
|
|
# ~/.gemini is simplest and also carries the cached account/settings. To
|
|
# reuse an existing host login without copying, set in .env:
|
|
# GEMINI_OAUTH_DIR=~/.gemini
|
|
# If unseeded, the path fail-opens to the DDG/Brave cascade and the
|
|
# entrypoint logs a warning. Only consumed when GEMINI_AUTH=oauth.
|
|
- ${GEMINI_OAUTH_DIR:-./docker/gemini-oauth}:/root/.gemini
|
|
# Operator instruction files. Every *.md here is appended to the main
|
|
# reply LLM's system prompt (filename order), read per turn so edits apply
|
|
# on the next reply without a rebuild/restart. Read-only; a project-
|
|
# relative path resolves identically on Linux and Windows Docker Desktop.
|
|
- ./agents:/app/agents:ro
|
|
|
|
volumes:
|
|
ollama_models:
|
|
javis_data:
|
|
whisper_cache:
|
|
piper_voices:
|