feat: replace MeloTTS with Coqui XTTS-v2 natural Korean voice

MeloTTS's single Korean speaker sounded non-native ("foreign accent"). Swap it
for Coqui XTTS-v2 with the built-in female studio speaker "Ana Florence"
(language ko), the natural voice used in earlier local runs.

- bridge/xtts_worker.py: new warm HTTP worker (own /opt/xtts venv), same
  /synth + /health contract and PCM16 output as the old melo worker
- docker/setup-xtts.sh: builds the venv with cu128 torch (Blackwell) + Coqui
  TTS and bakes the XTTS-v2 model offline. Pins transformers>=4.57,<5 (5.x
  removed isin_mps_friendly, breaking XTTS) and installs the [codec] extra
  (torch>=2.9 needs torchcodec) — both verified by a real host synth
- Dockerfile: replace the melo build layer with the xtts layer
- supervisord.conf: melo-worker -> xtts-worker, env passthrough for
  XTTS_DEVICE/SPEAKER/LANGUAGE (always set via compose defaults)
- bridge/server.py: default TTS_ENGINE=xtts, route to the xtts worker, generic
  worker-synth helper, neural-only fallback flag (XTTS_FALLBACK_PIPER)
- settings UI: engine dropdown xtts/piper, drop the dead melo_speed field, fix
  the supervisorctl restart target to xtts-worker
- compose/.env.example/README: XTTS_* vars, speaker/language knobs, remove melo
- remove bridge/melo_worker.py and docker/setup-melo.sh
- tests: xtts treated as multilingual (not English-only)

Verified on host: coqui-tts loads XTTS-v2 and synthesises Korean as
"Ana Florence" to a 16-bit mono 24kHz WAV.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

.env.example

@@ -34,18 +34,23 @@ WHISPER_DEVICE=cuda
 WHISPER_COMPUTE_TYPE=float16
 # Optional explicit Piper voice model (.onnx). If empty, the jarvis default is used.
 TTS_PIPER_MODEL_PATH=
-# TTS engine: "melo" (default) uses the MeloTTS Korean voice served by the warm
-# melo-worker (Korean speaker, speed 1.5). Set to "piper" to use Piper directly.
-TTS_ENGINE=melo
-# Melo-only by default: if MeloTTS synthesis fails the bridge returns no audio
+# TTS engine: "xtts" (default) uses the Coqui XTTS-v2 natural Korean voice
+# served by the warm xtts-worker. Set to "piper" to use the English Piper voice
+# directly. (MeloTTS was removed; "melo" only works with an out-of-band worker.)
+TTS_ENGINE=xtts
+# XTTS-v2 voice settings. Speaker is any built-in studio voice; "Ana Florence"
+# is a natural female voice. Language is the synthesis language (ko = Korean).
+XTTS_SPEAKER=Ana Florence
+XTTS_LANGUAGE=ko
+XTTS_DEVICE=cuda
+# Where the bridge reaches the in-container XTTS worker, and how long it waits
+# for a synthesis (XTTS is slower than Melo: ~1-2s/sentence on GPU).
+XTTS_WORKER_URL=http://127.0.0.1:8771
+XTTS_TIMEOUT=30
+# Neural-only by default: if XTTS synthesis fails the bridge returns no audio
 # rather than speaking Korean through the English Piper voice (which mangles it).
 # Set to 1 only if you explicitly want the Piper fallback.
-MELO_FALLBACK_PIPER=0
-# Where the bridge reaches the in-container MeloTTS worker, and how long it
-# waits for a synthesis. Speaking rate is set on the worker via MELO_SPEED.
-MELO_WORKER_URL=http://127.0.0.1:8770
-MELO_TIMEOUT=30
-MELO_SPEED=1.5
+XTTS_FALLBACK_PIPER=0
 
 # ---------------------------------------------------------------------------
 # Jarvis brain (Ollama-backed). In Docker these populate the rendered
@@ -74,6 +79,12 @@ WHISPER_MODEL=small
 # occasional trailing CJK fragment small models leak on free-form chat).
 OUTPUT_LANGUAGE=
 
+# Operator instruction folder: every *.md in this dir is appended to the main
+# reply LLM's system prompt (filename order), re-read each turn so edits apply
+# without a rebuild/restart. ./agents is bind-mounted here read-only; only
+# change this to relocate the folder inside the container. See README "운영자 지시문".
+AGENTS_DIR=/app/agents
+
 # ---------------------------------------------------------------------------
 # Docker desktop (VNC) — used only by the container image
 # ---------------------------------------------------------------------------
@@ -220,7 +231,7 @@ COMPOSE_FILE=docker-compose.yml:docker-compose.gpu-linux.yml
 # OLLAMA_CHAT_MODEL=qwen2.5:7b   # quality (needs ~5GB VRAM + whisper small)
 # OLLAMA_CHAT_MODEL=qwen2.5:3b   # speed (fits easily, faster on 8GB GPUs)
 # WHISPER_MODEL=small            # small frees VRAM for a bigger LLM; medium=more accurate
-# MELO_DEVICE=cuda               # cpu if no GPU on the bot host
+# XTTS_DEVICE=cuda               # cpu if no GPU on the bot host (XTTS is slow on CPU)
 
 # --- Settings web UI (http://localhost:8765/settings on the bot host) ---
 # To reach it, expose the bridge to the host loopback:

Dockerfile

@@ -65,18 +65,19 @@ RUN ls -d /opt/venv/lib/python*/site-packages/nvidia/cublas/lib \
         > /etc/ld.so.conf.d/nvidia-cu12.conf 2>/dev/null \
     && /sbin/ldconfig || true
 
-# --- MeloTTS Korean voice (separate /opt/melo py3.11 venv; see setup-melo.sh).
-#     Heavy layer (torch CPU + transformers + MeCab); placed before the app
-#     COPY so it stays cached across source-only changes. ---
-COPY docker/setup-melo.sh /app/docker/setup-melo.sh
+# --- Korean voice: Coqui XTTS-v2 (separate /opt/xtts py3.11 venv; see
+#     setup-xtts.sh). Natural female Korean ("Ana Florence"); replaces MeloTTS.
+#     Heavy layer (torch cu128 + Coqui TTS + the baked XTTS-v2 model); placed
+#     before the app COPY so it stays cached across source-only changes. ---
+COPY docker/setup-xtts.sh /app/docker/setup-xtts.sh
 # Strip CR before running: a Windows checkout (autocrlf) yields CRLF, which makes
-# bash read line 18 as `set -euxo pipefail\r` and abort with
-# "set: pipefail: invalid option name". .gitattributes pins *.sh to LF, but this
-# keeps the build working even on a not-yet-renormalised working tree.
-RUN sed -i 's/\r$//' /app/docker/setup-melo.sh && bash /app/docker/setup-melo.sh
+# bash read `set -euxo pipefail\r` and abort with "set: pipefail: invalid option
+# name". .gitattributes pins *.sh to LF, but this keeps the build working even on
+# a not-yet-renormalised working tree.
+RUN sed -i 's/\r$//' /app/docker/setup-xtts.sh && bash /app/docker/setup-xtts.sh
 
 # --- Human input + window management for the on-screen Chrome control tool.
-#     Placed AFTER the heavy melo layer so it doesn't bust that cache. xdotool
+#     Placed AFTER the heavy TTS layer so it doesn't bust that cache. xdotool
 #     injects real X pointer/keyboard events (visible cursor, char-by-char
 #     typing) into the broadcast; wmctrl lists/moves windows. ---
 RUN apt-get update && apt-get install -y --no-install-recommends \

README.md

@@ -69,7 +69,7 @@ docker compose -f docker-compose.yml -f docker-compose.gpu-linux.yml up -d --bui
 docker compose -f docker-compose.yml -f docker-compose.gpu-windows.yml up -d --build
 
 # ── GPU 없이 (CPU 전용 호스트) ──
-#   .env 에 WHISPER_DEVICE=cpu, MELO_DEVICE=cpu 를 넣고 베이스만 사용
+#   .env 에 WHISPER_DEVICE=cpu, XTTS_DEVICE=cpu 를 넣고 베이스만 사용
 docker compose up -d --build
 ```
 
@@ -113,7 +113,7 @@ docker compose up -d          # 유저봇이 로그인해 지정 음성채널에
 
 ### GPU 가속 (OS별)
 
-LLM(Ollama), Whisper STT, MeloTTS가 GPU에서 돕니다(env 기본 `WHISPER_DEVICE=cuda`, `MELO_DEVICE=cuda`). NVIDIA Blackwell(sm_120, 예: RTX 5050/5070Ti)에서 검증: 컨테이너 내 torch cu128 CUDA 동작, Ollama GPU 오프로드, faster-whisper float16, MeloTTS GPU 합성 모두 확인.
+LLM(Ollama), Whisper STT, XTTS-v2 TTS가 GPU에서 돕니다(env 기본 `WHISPER_DEVICE=cuda`, `XTTS_DEVICE=cuda`). NVIDIA Blackwell(sm_120, 예: RTX 5050/5070Ti)에서 검증: 컨테이너 내 torch cu128 CUDA 동작, Ollama GPU 오프로드, faster-whisper float16, XTTS-v2 GPU 합성 모두 확인.
 
 GPU는 위 "실행 — Docker"의 OS별 override 파일로 켜집니다. 호스트 사전 준비는 OS마다 다릅니다:
 
@@ -137,7 +137,7 @@ docker run --rm --device nvidia.com/gpu=all ubuntu nvidia-smi -L   # GPU 보이
 **공통:**
 
 - 모델: 베이스 compose 기본은 `qwen2.5:3b`(8GB VRAM에서 도구호출 안정적). 더 무겁게(`qwen2.5:7b`, `qwen3:8b` 등) 쓰려면 `.env`의 `OLLAMA_CHAT_MODEL` 변경.
-- **GPU가 없거나 인식 실패 시 자동으로 CPU 폴백**(Whisper)하므로 안전합니다. 명시적으로 CPU만 쓰려면 override 파일 없이 베이스만 올리고 `.env`에 `WHISPER_DEVICE=cpu`, `MELO_DEVICE=cpu`를 두세요.
+- **GPU가 없거나 인식 실패 시 자동으로 CPU 폴백**(Whisper)하므로 안전합니다. 명시적으로 CPU만 쓰려면 override 파일 없이 베이스만 올리고 `.env`에 `WHISPER_DEVICE=cpu`, `XTTS_DEVICE=cpu`를 두세요.
 
 - 데이터(메모리 DB), Whisper 캐시, Piper 음성은 named volume에 영속됩니다.
 - 셀프봇 영상 송출 의존성은 이미지에 기본 포함하지 않습니다. 쓰려면 컨테이너에서 `cd /app/bot && bun add discord.js-selfbot-v13 @dank074/discord-video-stream` 후 재시작(또는 Dockerfile에 추가).
@@ -243,10 +243,22 @@ cd bot; bun run register; bun run start   # 창 2: (일반 봇이면) 슬래시
 - `BRIDGE_URL` — 봇이 호출할 브릿지 주소 (기본 `http://127.0.0.1:8765`)
 - `STREAM_BACKEND`, `DISCORD_SELFBOT_TOKEN`, `NOVNC_URL` — 화면 송출
 - `VNC_DISPLAY=:1`, `VNC_RESOLUTION`, `VNC_FRAMERATE`, `VNC_BITRATE_KBPS` — 캡처
-- `WHISPER_DEVICE/COMPUTE_TYPE`, `MELO_DEVICE` — GPU 호스트면 `cuda`/`float16`, CPU 전용이면 `cpu`(GPU 자체는 OS별 override compose 파일로 켬)
+- `WHISPER_DEVICE/COMPUTE_TYPE`, `XTTS_DEVICE` — GPU 호스트면 `cuda`/`float16`, CPU 전용이면 `cpu`(GPU 자체는 OS별 override compose 파일로 켬)
 - `OLLAMA_CHAT_MODEL` — 두뇌 LLM (기본 `qwen2.5:3b`)
 - `COMPOSE_FILE` — OS별 GPU override를 매번 `-f`로 안 치고 자동 적용 (위 "실행 — Docker" 참고)
 - `output_language` — 출력 언어 고정(비우면 사용자 언어). 설정 웹 UI(`/settings`)에서 바꾸면 env 기본값보다 우선하며 컨테이너 재생성 후에도 유지됩니다.
+- `AGENTS_DIR` — 운영자 지시문 폴더(기본 `/app/agents`, `./agents`가 read-only로 마운트됨). 아래 "운영자 지시문" 참고.
+
+---
+
+## 운영자 지시문 (`agents/*.md`)
+
+`agents/` 폴더에 마크다운 파일을 넣으면 그 내용이 어시스턴트의 메인 답변 시스템 프롬프트 뒤에 그대로 추가됩니다. 페르소나(집사 성격)는 그대로 두고 규칙·말투·금칙어 등을 덧붙일 때 쓰세요.
+
+- `agents/` 안의 모든 `*.md`를 **파일명 순서**로 이어 붙입니다. 순서를 정하려면 `00-tone.md`, `10-rules.md`처럼 숫자 접두사를 쓰세요.
+- **매 답변마다 다시 읽습니다.** 파일을 저장하면 다음 발화부터 바로 반영되며, 재빌드/재시작이 필요 없습니다(폴더가 read-only로 마운트됨).
+- 폴더가 없거나 비어 있으면 아무 일도 일어나지 않습니다(fail-open).
+- `agents/example.md.sample`을 `rules.md` 등 `*.md`로 복사해서 시작하세요. `.sample` 파일은 로드되지 않습니다.
 
 ---
 

agents/example.md.sample

@@ -0,0 +1,15 @@
+# Operator instruction file (example)
+#
+# HOW TO USE: copy or rename this file to anything ending in `.md`
+# (e.g. `rules.md`). Every `*.md` in this folder is appended to the assistant's
+# main reply system prompt, in filename order — use number prefixes like
+# `00-tone.md`, `10-rules.md` to control ordering. Edits take effect on the
+# NEXT reply; no rebuild or restart is needed (the folder is read per turn).
+#
+# Files ending in `.sample` (like this one) are ignored, so this template never
+# affects replies until you rename it to `*.md`.
+#
+# Everything below a heading is treated as plain instruction text for the LLM.
+
+Always keep replies under two sentences.
+When the user asks about deployment, mention the relevant docker compose command.

bridge/server.py

@@ -87,12 +87,13 @@ VAD_MIN_SPEECH_MS = int(os.environ.get("VAD_MIN_SPEECH_MS", "200"))
 # Korean phrase decoded as Chinese) and shaves a little latency. Empty = auto.
 STT_LANGUAGE = os.environ.get("STT_LANGUAGE", "ko").strip() or None
 
-# TTS engine: "melo" (MeloTTS Korean speaker, the warm worker) is the primary
-# voice; Piper is kept as a fallback if the worker is unreachable. Set
-# TTS_ENGINE=piper to disable MeloTTS entirely.
+# TTS engine: "xtts" (Coqui XTTS-v2 natural Korean voice, the warm worker) is
+# the primary voice; Piper is kept as a fallback only if explicitly enabled. Set
+# TTS_ENGINE=piper to disable the neural Korean voice entirely. "melo" is still
+# accepted for backward compatibility but is no longer built into the image.
 def _tts_engine_setting() -> str:
     """TTS engine: settings-UI value (runtime config JSON) wins, else env, else
-    melo. Read at startup; the settings UI restarts the bridge on apply."""
+    xtts. Read at startup; the settings UI restarts the bridge on apply."""
     try:
         _cp = os.environ.get("JARVIS_CONFIG_PATH", "/app/config/jarvis.json")
         _v = json.loads(open(_cp, encoding="utf-8").read()).get("tts_engine")
@@ -100,17 +101,29 @@ def _tts_engine_setting() -> str:
             return str(_v).strip().lower()
     except Exception:
         pass
-    return os.environ.get("TTS_ENGINE", "melo").strip().lower()
+    return os.environ.get("TTS_ENGINE", "xtts").strip().lower()
 
 
 TTS_ENGINE = _tts_engine_setting()
+# Coqui XTTS-v2 worker (the natural Korean voice).
+XTTS_WORKER_URL = os.environ.get("XTTS_WORKER_URL", "http://127.0.0.1:8771")
+XTTS_TIMEOUT = float(os.environ.get("XTTS_TIMEOUT", "30"))
+# Legacy MeloTTS worker (no longer built into the image; kept for back-compat
+# if someone runs an old worker out-of-band).
 MELO_WORKER_URL = os.environ.get("MELO_WORKER_URL", "http://127.0.0.1:8770")
 MELO_TIMEOUT = float(os.environ.get("MELO_TIMEOUT", "30"))
-# When MeloTTS is the engine, do NOT silently fall back to the English Piper
-# voice on failure: speaking Korean text through an English voice produces
-# mangled audio. Default is melo-only (return no audio on failure); set
-# MELO_FALLBACK_PIPER=1 to opt into the Piper fallback.
-MELO_FALLBACK_PIPER = os.environ.get("MELO_FALLBACK_PIPER", "0") in ("1", "true", "True", "yes", "on")
+# Do NOT silently fall back to the English Piper voice on a neural-voice failure:
+# speaking Korean text through an English voice produces mangled audio. Default
+# is neural-only (return no audio on failure); set XTTS_FALLBACK_PIPER=1 (or the
+# legacy MELO_FALLBACK_PIPER=1) to opt into the Piper fallback.
+def _truthy_env(*names: str) -> bool:
+    for _n in names:
+        if os.environ.get(_n, "").strip().lower() in ("1", "true", "yes", "on"):
+            return True
+    return False
+
+
+NEURAL_FALLBACK_PIPER = _truthy_env("XTTS_FALLBACK_PIPER", "MELO_FALLBACK_PIPER")
 
 # ---------------------------------------------------------------------------
 # Lazy singletons. The first request pays the model-load cost; afterwards the
@@ -150,12 +163,17 @@ def _ensure_brain():
             compute = os.environ.get("WHISPER_COMPUTE_TYPE", "auto")
             try:
                 whisper = WhisperModel(cfg.whisper_model, device=device, compute_type=compute)
+                # Log the device actually resolved by CTranslate2 (device="auto"
+                # picks cuda when available) so a silent CPU load is visible.
+                resolved = str(getattr(getattr(whisper, "model", None), "device", device)).lower()
+                print(f"[bridge] whisper loaded on {resolved} (compute={compute})", flush=True)
             except Exception as ge:
                 # GPU not available / unsupported -> fall back to CPU so the
                 # bridge still works without a GPU passed to the container.
                 if device != "cpu":
                     print(f"[bridge] whisper device='{device}' failed ({ge}); falling back to CPU", flush=True)
                     whisper = WhisperModel(cfg.whisper_model, device="cpu", compute_type="int8")
+                    print("[bridge] whisper loaded on cpu (compute=int8)", flush=True)
                 else:
                     raise
 
@@ -297,27 +315,38 @@ def _coerce_bool(value) -> Optional[bool]:
     return str(value).strip().lower() in ("1", "true", "yes", "on")
 
 
-def _melo_synthesize(text: str) -> Optional[bytes]:
-    """Synthesise via the warm MeloTTS worker (separate /opt/melo venv, Korean
-    speaker @ speed 1.5). Returns a 16-bit PCM WAV, or None on any failure so
-    the caller can fall back to Piper."""
+def _worker_synthesize(name: str, url: str, timeout: float, text: str) -> Optional[bytes]:
+    """POST text to a warm TTS worker's /synth and return its WAV bytes, or None
+    on any failure so the caller can decide whether to fall back."""
     import urllib.request
 
     try:
         req = urllib.request.Request(
-            f"{MELO_WORKER_URL}/synth",
+            f"{url}/synth",
             data=json.dumps({"text": text}).encode("utf-8"),
             headers={"Content-Type": "application/json"},
         )
-        with urllib.request.urlopen(req, timeout=MELO_TIMEOUT) as resp:
+        with urllib.request.urlopen(req, timeout=timeout) as resp:
             if resp.status == 200:
                 return resp.read()
-            print(f"[bridge] melo worker HTTP {resp.status}", flush=True)
+            print(f"[bridge] {name} worker HTTP {resp.status}", flush=True)
     except Exception as e:  # pragma: no cover - worker may be down
-        print(f"[bridge] melo worker unreachable: {e}", flush=True)
+        print(f"[bridge] {name} worker unreachable: {e}", flush=True)
     return None
 
 
+def _xtts_synthesize(text: str) -> Optional[bytes]:
+    """Synthesise via the warm Coqui XTTS-v2 worker (separate /opt/xtts venv,
+    natural female Korean). Returns a 16-bit PCM WAV, or None on failure."""
+    return _worker_synthesize("xtts", XTTS_WORKER_URL, XTTS_TIMEOUT, text)
+
+
+def _melo_synthesize(text: str) -> Optional[bytes]:
+    """Legacy: synthesise via a MeloTTS worker if one is running out-of-band.
+    Returns a 16-bit PCM WAV, or None on any failure."""
+    return _worker_synthesize("melo", MELO_WORKER_URL, MELO_TIMEOUT, text)
+
+
 def _piper_synthesize(text: str) -> Optional[bytes]:
     """Fallback: synthesise with Piper (English voice). Returns WAV bytes."""
     _ensure_piper()
@@ -344,11 +373,12 @@ def _tts_ready() -> bool:
     """
     if not TTS_ENABLED:
         return True
-    if TTS_ENGINE == "melo":
+    _worker_health = {"xtts": XTTS_WORKER_URL, "melo": MELO_WORKER_URL}.get(TTS_ENGINE)
+    if _worker_health:
         import urllib.request
 
         try:
-            with urllib.request.urlopen(f"{MELO_WORKER_URL}/health", timeout=2) as resp:
+            with urllib.request.urlopen(f"{_worker_health}/health", timeout=2) as resp:
                 return resp.status == 200
         except Exception:
             return False
@@ -356,20 +386,24 @@ def _tts_ready() -> bool:
 
 
 def synthesize(text: str) -> Optional[bytes]:
-    """Synthesize text to a 16-bit PCM WAV. The primary voice is MeloTTS
-    (Korean speaker, speed 1.5) served by the warm melo worker; Piper is a
-    fallback if the worker is unavailable. Returns None if TTS is off."""
+    """Synthesize text to a 16-bit PCM WAV. The primary voice is Coqui XTTS-v2
+    (natural female Korean) served by the warm xtts worker; Piper is used only
+    when explicitly enabled as a fallback. Returns None if TTS is off."""
     if not TTS_ENABLED or not text.strip():
         return None
-    if TTS_ENGINE == "melo":
-        audio = _melo_synthesize(text)
+    _neural = {"xtts": _xtts_synthesize, "melo": _melo_synthesize}.get(TTS_ENGINE)
+    if _neural is not None:
+        audio = _neural(text)
         if audio:
             return audio
-        if not MELO_FALLBACK_PIPER:
-            # Melo-only: better silent than mangled English for Korean text.
-            print("[bridge] melo synth failed; no audio (Piper fallback disabled)", flush=True)
+        if not NEURAL_FALLBACK_PIPER:
+            # Neural-only: better silent than mangled English for Korean text.
+            print(
+                f"[bridge] {TTS_ENGINE} synth failed; no audio (Piper fallback disabled)",
+                flush=True,
+            )
             return None
-        print("[bridge] melo synth failed; falling back to Piper", flush=True)
+        print(f"[bridge] {TTS_ENGINE} synth failed; falling back to Piper", flush=True)
     return _piper_synthesize(text)
 
 

bridge/settings_web.py

@@ -22,8 +22,7 @@ from typing import Any, Dict
 FIELDS = [
     ("ollama_chat_model", "LLM 모델", "model"),
     ("whisper_model", "STT(Whisper) 모델", "select:tiny,base,small,medium,large,large-v3"),
-    ("tts_engine", "TTS 엔진", "select:melo,piper"),
-    ("melo_speed", "TTS 속도 (MeloTTS)", "number:0.5:2.5:0.1"),
+    ("tts_engine", "TTS 엔진", "select:xtts,piper"),
     ("output_language", "출력 언어 (비우면 사용자 언어)", "text"),
     ("llm_thinking_enabled", "LLM 사고(thinking) 모드", "bool"),
     ("agentic_max_turns", "에이전트 최대 반복", "number:1:12:1"),
@@ -54,9 +53,7 @@ def _current() -> Dict[str, Any]:
     cfg = _read_config()
     out: Dict[str, Any] = {}
     for k in _KEYS:
-        if k == "melo_speed":
-            out[k] = cfg.get("melo_speed", os.environ.get("MELO_SPEED", "1.5"))
-        elif k == "output_language":
+        if k == "output_language":
             out[k] = cfg.get("output_language", os.environ.get("OUTPUT_LANGUAGE", ""))
         else:
             out[k] = cfg.get(k, "")
@@ -78,12 +75,7 @@ def _coerce(updates: Dict[str, Any]) -> Dict[str, Any]:
     for k, v in updates.items():
         if k not in _KEYS:
             continue
-        if k == "melo_speed":
-            try:
-                v = float(v)
-            except (TypeError, ValueError):
-                continue
-        elif k == "agentic_max_turns":
+        if k == "agentic_max_turns":
             try:
                 v = int(v)
             except (TypeError, ValueError):
@@ -114,12 +106,12 @@ def _save(updates: Dict[str, Any]) -> None:
 
 
 def _apply() -> str:
-    # Restart melo + bridge AFTER this response is sent. Detached (new session)
-    # so the bridge being killed mid-restart doesn't drop the restart itself,
-    # and the HTTP client still receives this response.
+    # Restart the TTS worker + bridge AFTER this response is sent. Detached (new
+    # session) so the bridge being killed mid-restart doesn't drop the restart
+    # itself, and the HTTP client still receives this response.
     try:
         subprocess.Popen(
-            ["sh", "-c", "sleep 1; supervisorctl restart melo-worker bridge"],
+            ["sh", "-c", "sleep 1; supervisorctl restart xtts-worker bridge"],
             start_new_session=True,
         )
         return "1초 후 브리지/TTS 워커가 재시작되어 반영됩니다."

bridge/xtts_worker.py

@@ -1,25 +1,30 @@
 """
-MeloTTS worker
-==============
+XTTS worker
+===========
 
-A tiny, dependency-light HTTP service that keeps a MeloTTS voice warm and
-synthesises speech on demand. It runs in its OWN Python venv (``/opt/melo`` in
-the container) so the heavy MeloTTS/torch/transformers stack stays isolated
-from the slim brain-bridge venv (which pins ``numpy<2`` for faster-whisper).
+A tiny HTTP service that keeps a Coqui XTTS-v2 voice warm and synthesises
+speech on demand. It mirrors ``melo_worker.py`` (same ``/synth`` + ``/health``
+contract, same PCM16 WAV output) so the bridge can talk to either worker the
+same way.
 
-The bridge's ``synthesize()`` POSTs ``{"text": "..."}`` here and gets back a
-16-bit PCM WAV. The MeloTTS model is loaded once at startup and reused, so each
-request only pays inference cost, not model-load cost.
+XTTS-v2 is a natural, multilingual neural voice. The default speaker is the
+built-in female studio voice "Ana Florence" speaking Korean — the voice this
+deployment uses in place of MeloTTS. No reference WAV is needed for the
+built-in studio speakers.
+
+It runs in its OWN Python venv (``/opt/xtts`` in the container) so the heavy
+Coqui TTS / torch stack stays isolated from the slim brain-bridge venv.
 
 Config (env):
-    MELO_WORKER_HOST   bind host          (default 127.0.0.1)
-    MELO_WORKER_PORT   bind port          (default 8770)
-    MELO_LANGUAGE      MeloTTS language   (default KR)
-    MELO_SPEED         speaking rate      (default 1.5  -> the approved "150")
-    MELO_DEVICE        torch device       (default cpu)
+    XTTS_WORKER_HOST   bind host            (default 127.0.0.1)
+    XTTS_WORKER_PORT   bind port            (default 8771)
+    XTTS_MODEL         Coqui model id       (default tts_models/multilingual/multi-dataset/xtts_v2)
+    XTTS_SPEAKER       built-in speaker     (default "Ana Florence")
+    XTTS_LANGUAGE      synthesis language   (default ko)
+    XTTS_DEVICE        torch device         (default cpu; compose sets cuda)
 
 Run:
-    /opt/melo/bin/python -m bridge.melo_worker
+    /opt/xtts/bin/python -m bridge.xtts_worker
 """
 
 from __future__ import annotations
@@ -33,94 +38,72 @@ import threading
 import wave
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 
-HOST = os.environ.get("MELO_WORKER_HOST", "127.0.0.1")
-PORT = int(os.environ.get("MELO_WORKER_PORT", "8770"))
-LANGUAGE = os.environ.get("MELO_LANGUAGE", "KR")
+# XTTS-v2 is gated behind a one-time license prompt; agreeing here keeps the
+# load non-interactive in a container. XTTS-v2 is non-commercial (CPML).
+os.environ.setdefault("COQUI_TOS_AGREED", "1")
 
+HOST = os.environ.get("XTTS_WORKER_HOST", "127.0.0.1")
+PORT = int(os.environ.get("XTTS_WORKER_PORT", "8771"))
+MODEL = os.environ.get("XTTS_MODEL", "tts_models/multilingual/multi-dataset/xtts_v2")
+SPEAKER = os.environ.get("XTTS_SPEAKER", "Ana Florence")
+LANGUAGE = os.environ.get("XTTS_LANGUAGE", "ko")
+DEVICE = os.environ.get("XTTS_DEVICE", "cpu")
 
-def _resolve_speed() -> float:
-    """Speaking rate: the settings-UI value (runtime config JSON) wins, else the
-    MELO_SPEED env, else 1.5. Read at startup; the settings UI restarts this
-    worker on apply so a new value takes effect."""
-    try:
-        cp = os.environ.get("JARVIS_CONFIG_PATH", "/app/config/jarvis.json")
-        v = json.loads(open(cp, encoding="utf-8").read()).get("melo_speed")
-        if v is not None:
-            return float(v)
-    except Exception:
-        pass
-    try:
-        return float(os.environ.get("MELO_SPEED", "1.5"))
-    except ValueError:
-        return 1.5
-
-
-SPEED = _resolve_speed()
-DEVICE = os.environ.get("MELO_DEVICE", "cpu")
-
-# Model + speaker id are loaded once, guarded by a lock because MeloTTS
-# inference is not guaranteed thread-safe.
+# Model is loaded once, guarded by a lock because TTS inference is not
+# guaranteed thread-safe.
 _model = None
-_speaker_id = None
 _model_lock = threading.Lock()
 _load_error: str | None = None
 
 
 def _ensure_model() -> None:
-    global _model, _speaker_id, _load_error
+    global _model, _load_error
     if _model is not None or _load_error is not None:
         return
     with _model_lock:
         if _model is not None or _load_error is not None:
             return
         try:
-            from melo.api import TTS  # type: ignore
+            from TTS.api import TTS  # type: ignore
 
-            model = TTS(language=LANGUAGE, device=DEVICE)
-            # spk2id is a melo HParams object (dict-like, supports __getitem__,
-            # __contains__, keys) but NOT .get(). The KR model exposes a single
-            # 'KR' speaker; fall back to the first id for other languages.
-            spk_map = model.hps.data.spk2id
-            keys = list(spk_map.keys())
-            speaker_id = spk_map[LANGUAGE] if LANGUAGE in spk_map else spk_map[keys[0]]
+            model = TTS(MODEL).to(DEVICE)
             _model = model
-            _speaker_id = speaker_id
-            # Warm the GPU once at load: the first CUDA synth pays a one-off
-            # kernel-init cost (~5s) that would otherwise land on the user's
-            # first reply. A throwaway synth here moves it to startup. No-op
-            # cost on CPU.
+            # Warm once: the first GPU synth pays a one-off kernel-init cost
+            # that would otherwise land on the user's first reply.
             try:
                 with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as _wt:
                     _wp = _wt.name
-                model.tts_to_file("워밍업", speaker_id, _wp, speed=SPEED)
+                model.tts_to_file(
+                    text="워밍업", speaker=SPEAKER, language=LANGUAGE, file_path=_wp
+                )
                 try:
                     os.unlink(_wp)
                 except OSError:
                     pass
             except Exception as _we:  # pragma: no cover
-                print(f"[melo-worker] warmup synth skipped: {_we}", flush=True)
+                print(f"[xtts-worker] warmup synth skipped: {_we}", flush=True)
             print(
-                f"[melo-worker] ready (lang={LANGUAGE} speed={SPEED} "
-                f"device={DEVICE} speakers={list(spk_map.keys())})",
+                f"[xtts-worker] ready (model={MODEL} speaker={SPEAKER!r} "
+                f"language={LANGUAGE} device={DEVICE})",
                 flush=True,
             )
         except Exception as e:  # pragma: no cover - depends on local model files
             _load_error = f"{type(e).__name__}: {e}"
-            print(f"[melo-worker] model load FAILED: {_load_error}", flush=True)
+            print(f"[xtts-worker] model load FAILED: {_load_error}", flush=True)
 
 
 def _synthesize(text: str) -> bytes:
     """Synthesise ``text`` to a 16-bit PCM WAV (bytes)."""
     _ensure_model()
     if _model is None:
-        raise RuntimeError(_load_error or "melo model unavailable")
-    # MeloTTS writes to a file via soundfile; render to a container-disk temp
-    # file (NOT tmpfs), read it back, then drop it.
+        raise RuntimeError(_load_error or "xtts model unavailable")
     with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
         tmp_path = tmp.name
     try:
         with _model_lock:
-            _model.tts_to_file(text, _speaker_id, tmp_path, speed=SPEED)
+            _model.tts_to_file(
+                text=text, speaker=SPEAKER, language=LANGUAGE, file_path=tmp_path
+            )
         with open(tmp_path, "rb") as f:
             raw = f.read()
     finally:
@@ -132,16 +115,15 @@ def _synthesize(text: str) -> bytes:
 
 
 def _ensure_pcm16_wav(raw: bytes) -> bytes:
-    """Guarantee a 16-bit PCM WAV. MeloTTS/soundfile usually emit float WAVs;
-    the Discord playback path (ffmpeg) tolerates both, but we normalise to
-    PCM16 so the contract matches the previous Piper output."""
+    """Guarantee a 16-bit PCM WAV. Coqui writes float/other WAVs; the Discord
+    playback path tolerates both, but we normalise to PCM16 so the contract
+    matches the previous Melo/Piper output (mono, file's own sample rate)."""
     try:
         with wave.open(io.BytesIO(raw), "rb") as wf:
             if wf.getsampwidth() == 2:
                 return raw  # already PCM16
     except wave.Error:
         pass
-    # Non-PCM16 (e.g. float) — convert with soundfile if available.
     try:
         import numpy as np
         import soundfile as sf
@@ -159,7 +141,7 @@ def _ensure_pcm16_wav(raw: bytes) -> bytes:
             wf.writeframes(pcm)
         return buf.getvalue()
     except Exception:
-        return raw  # last resort: hand back whatever MeloTTS produced
+        return raw  # last resort: hand back whatever XTTS produced
 
 
 class _Handler(BaseHTTPRequestHandler):
@@ -212,7 +194,7 @@ def main() -> int:
     # Warm the model at startup so the first Discord turn isn't slow.
     _ensure_model()
     server = ThreadingHTTPServer((HOST, PORT), _Handler)
-    print(f"[melo-worker] listening on http://{HOST}:{PORT}", flush=True)
+    print(f"[xtts-worker] listening on http://{HOST}:{PORT}", flush=True)
     try:
         server.serve_forever()
     except KeyboardInterrupt:

docker-compose.yml

@@ -66,9 +66,15 @@ services:
       WHISPER_MODEL: ${WHISPER_MODEL:-medium}
       WHISPER_DEVICE: ${WHISPER_DEVICE:-cuda}
       WHISPER_COMPUTE_TYPE: ${WHISPER_COMPUTE_TYPE:-float16}
-      # MeloTTS on the GPU (cu128 torch baked by docker/setup-melo.sh). CPU synth
-      # serialised under load and pushed TTS to 7-8s; GPU does ~0.3s/sentence.
-      MELO_DEVICE: ${MELO_DEVICE:-cuda}
+      # Coqui XTTS-v2 (natural female Korean voice, replaces MeloTTS) on the GPU
+      # (cu128 torch baked by docker/setup-xtts.sh). Set here WITH DEFAULTS so
+      # supervisord's %(ENV_XTTS_*)s passthrough always resolves and an .env
+      # override actually reaches the xtts-worker.
+      XTTS_DEVICE: ${XTTS_DEVICE:-cuda}
+      # Built-in studio speaker (female). Other options include "Daisy Studious",
+      # "Sofia Hellen", "Alma María", etc. — any XTTS-v2 studio speaker name.
+      XTTS_SPEAKER: ${XTTS_SPEAKER:-Ana Florence}
+      XTTS_LANGUAGE: ${XTTS_LANGUAGE:-ko}
       # Optional single-language lock for replies (empty = user's own language).
       OUTPUT_LANGUAGE: ${OUTPUT_LANGUAGE:-ko}
       # Drop the pre-loop planner LLM call to cut voice-reply latency on small
@@ -97,6 +103,10 @@ services:
       BROWSER_CONTROL_BIND: ${BROWSER_CONTROL_BIND:-0.0.0.0}
       BROWSER_CONTROL_PORT: ${BROWSER_CONTROL_PORT:-8777}
       BROWSER_CONTROL_URL: ${BROWSER_CONTROL_URL:-}
+      # Folder of operator *.md instruction files appended to the main reply
+      # LLM's system prompt. Bind-mounted from ./agents below; override only to
+      # relocate it inside the container.
+      AGENTS_DIR: ${AGENTS_DIR:-/app/agents}
     # No hard depends_on ollama: a browser-host (`docker compose up -d javis`)
     # must NOT pull in Ollama. Full/bot layouts start it with a plain
     # `docker compose up -d` (all services); the bridge tolerates Ollama warming
@@ -149,6 +159,11 @@ services:
       # If unseeded, the path fail-opens to the DDG/Brave cascade and the
       # entrypoint logs a warning. Only consumed when GEMINI_AUTH=oauth.
       - ${GEMINI_OAUTH_DIR:-./docker/gemini-oauth}:/root/.gemini
+      # Operator instruction files. Every *.md here is appended to the main
+      # reply LLM's system prompt (filename order), read per turn so edits apply
+      # on the next reply without a rebuild/restart. Read-only; a project-
+      # relative path resolves identically on Linux and Windows Docker Desktop.
+      - ./agents:/app/agents:ro
 
 volumes:
   ollama_models:

docker/setup-melo.sh

@@ -1,80 +0,0 @@
-#!/usr/bin/env bash
-# ============================================================================
-# Install a dedicated MeloTTS (Korean voice) venv at /opt/melo.
-#
-# Why a SEPARATE venv (not the brain-bridge /opt/venv):
-#   - MeloTTS pins old deps (transformers 4.27.4 / tokenizers 0.13.3 / fugashi)
-#     whose binary wheels exist only for cp311, so we use python3.11 here even
-#     though the image's default interpreter is 3.12.
-#   - It isolates the heavy torch/transformers stack from the slim bridge env,
-#     which pins numpy<2 for faster-whisper.
-#
-# torch is the CUDA (cu128) build so MeloTTS runs on the GPU alongside Ollama +
-# Whisper. CPU synth serialised under concurrent load (whisper STT + bot) and
-# blew TTS up to 7-8s per reply; on the GPU a sentence synthesises in ~0.3s.
-# cu128 is the Blackwell (sm_120) wheel verified on this host's RTX 5050.
-# The worker selects the device via MELO_DEVICE=cuda (compose).
-# ============================================================================
-set -euxo pipefail
-
-export DEBIAN_FRONTEND=noninteractive
-
-apt-get update
-# Build deps for fugashi / mecab-python3 + a system MeCab dict, plus python3.11.
-apt-get install -y --no-install-recommends \
-    software-properties-common build-essential pkg-config swig \
-    libmecab-dev mecab mecab-ipadic-utf8
-add-apt-repository -y ppa:deadsnakes/ppa
-apt-get update
-apt-get install -y --no-install-recommends python3.11 python3.11-venv python3.11-dev
-rm -rf /var/lib/apt/lists/*
-
-python3.11 -m venv /opt/melo
-/opt/melo/bin/pip install --no-cache-dir --upgrade pip wheel setuptools
-
-# CUDA (cu128) torch first, so MeloTTS's unpinned `torch` dep is already
-# satisfied with the GPU build. Pinned to the Blackwell-verified versions
-# (2.11.0+cu128) for reproducible rebuilds.
-/opt/melo/bin/pip install --no-cache-dir torch==2.11.0+cu128 torchaudio==2.11.0+cu128 \
-    --index-url https://download.pytorch.org/whl/cu128
-
-# MeloTTS from GitHub. The PyPI sdist is broken (its setup.py reads a
-# requirements.txt that is not shipped in the sdist), so install from the repo.
-# Pinned to a commit (not refs/heads/main) so rebuilds are reproducible.
-/opt/melo/bin/pip install --no-cache-dir \
-    "https://github.com/myshell-ai/MeloTTS/archive/209145371cff8fc3bd60d7be902ea69cbdb7965a.tar.gz"
-
-# Korean g2p backend. MeloTTS otherwise tries to pip-install this on the first
-# Korean request, which fails in a network-isolated container at runtime.
-/opt/melo/bin/pip install --no-cache-dir python-mecab-ko python-mecab-ko-dic
-
-# Remove the full `unidic` package (its dictionary is never downloaded, only a
-# stub) so mecab-python3 falls back to the bundled `unidic_lite` dict. Without
-# this, importing melo's Japanese module fails with a missing-mecabrc error.
-/opt/melo/bin/pip uninstall -y unidic || true
-
-# Pre-cache every model asset MeloTTS pulls at runtime, so the worker starts
-# offline and the first Discord turn pays no download cost. Importing melo.api
-# fetches the Japanese (tohoku-nlp/bert-base-japanese-v3) and Korean
-# (kykim/bert-kor-base) BERT tokenizers plus nltk g2p data; loading the KR voice
-# downloads the OpenVoice KR config+checkpoint, and a real synth pulls the
-# Korean BERT weights. All of these go through huggingface_hub.
-#
-# CRITICAL: at runtime docker-compose mounts the `whisper_cache` named volume
-# over /root/.cache/huggingface (for faster-whisper). That volume would SHADOW
-# anything baked into the default HF cache, so we pin the melo worker to a
-# DEDICATED, non-volume cache dir (/opt/melo-cache) here AND in supervisord, and
-# warm it once. nltk_data (/root/nltk_data) is not volume-mounted so it stays.
-export HF_HOME=/opt/melo-cache
-mkdir -p "$HF_HOME"
-MELO_LANGUAGE=KR HF_HOME=/opt/melo-cache /opt/melo/bin/python - <<'PY'
-import tempfile
-from melo.api import TTS
-
-model = TTS(language="KR", device="cpu")
-out = tempfile.mktemp(suffix=".wav")
-model.tts_to_file("초기화 워밍업입니다.", model.hps.data.spk2id["KR"], out, speed=1.5)
-print("[setup-melo] warm-up KR synth OK ->", out)
-PY
-
-echo "[setup-melo] MeloTTS venv ready at /opt/melo"

docker/setup-xtts.sh

@@ -0,0 +1,72 @@
+#!/usr/bin/env bash
+# ============================================================================
+# Install a dedicated Coqui XTTS-v2 (natural Korean voice) venv at /opt/xtts.
+#
+# Why a SEPARATE venv (not the brain-bridge /opt/venv or /opt/melo):
+#   - Coqui TTS pulls its own heavy torch/transformers stack; isolating it keeps
+#     the slim bridge env (numpy<2 for faster-whisper) untouched.
+#   - We use python3.11 (installed for the melo layer) because Coqui ships cp311
+#     wheels and torch cu128 is available for it.
+#
+# torch is the CUDA (cu128) build so XTTS runs on the GPU alongside Ollama +
+# Whisper. cu128 is the Blackwell (sm_120) wheel verified on this host.
+# The worker selects the device via XTTS_DEVICE=cuda (compose).
+#
+# XTTS-v2 is non-commercial (Coqui Public Model License). COQUI_TOS_AGREED=1
+# accepts it non-interactively so the model can load in a headless container.
+# ============================================================================
+set -euxo pipefail
+
+export DEBIAN_FRONTEND=noninteractive
+export COQUI_TOS_AGREED=1
+
+# Install python3.11 if not already present, so this layer is self-contained.
+if ! command -v python3.11 >/dev/null 2>&1; then
+    apt-get update
+    apt-get install -y --no-install-recommends software-properties-common
+    add-apt-repository -y ppa:deadsnakes/ppa
+    apt-get update
+    apt-get install -y --no-install-recommends python3.11 python3.11-venv python3.11-dev
+    rm -rf /var/lib/apt/lists/*
+fi
+
+python3.11 -m venv /opt/xtts
+/opt/xtts/bin/pip install --no-cache-dir --upgrade pip wheel setuptools
+
+# CUDA (cu128) torch first so Coqui's `torch` dep is satisfied with the GPU
+# build. Pinned to the Blackwell-verified versions for reproducible rebuilds.
+/opt/xtts/bin/pip install --no-cache-dir torch==2.11.0+cu128 torchaudio==2.11.0+cu128 \
+    --index-url https://download.pytorch.org/whl/cu128
+
+# Coqui TTS (maintained fork; provides the `TTS` package and XTTS-v2). The
+# [codec] extra pulls torchcodec, which torch >=2.9 requires for audio IO
+# (without it the import fails with TORCHCODEC_IMPORT_ERROR). torchcodec also
+# needs the system FFmpeg shared libs, which are present (ffmpeg apt package).
+/opt/xtts/bin/pip install --no-cache-dir "coqui-tts[codec]"
+
+# Pin transformers to the 4.57+ / <5 range. coqui-tts requires >=4.57 but does
+# NOT cap the upper bound, and transformers 5.x removed `isin_mps_friendly`
+# (used by XTTS's tortoise layer), so an unpinned install pulls 5.x and the
+# model import fails with "cannot import name 'isin_mps_friendly'". Pin <5.
+/opt/xtts/bin/pip install --no-cache-dir "transformers>=4.57,<5"
+
+# Pre-bake the XTTS-v2 model so the worker starts offline and the first Discord
+# turn pays no download cost. The model is cached under TTS_HOME; we pin a
+# DEDICATED, non-volume dir (/opt/xtts-cache) AND set it in supervisord, because
+# runtime volume mounts (whisper_cache over /root/.cache) must not shadow it.
+export TTS_HOME=/opt/xtts-cache
+mkdir -p "$TTS_HOME"
+COQUI_TOS_AGREED=1 TTS_HOME=/opt/xtts-cache XTTS_SPEAKER="Ana Florence" \
+    /opt/xtts/bin/python - <<'PY'
+import os
+os.environ["COQUI_TOS_AGREED"] = "1"
+from TTS.api import TTS
+
+speaker = os.environ.get("XTTS_SPEAKER", "Ana Florence")
+model = TTS("tts_models/multilingual/multi-dataset/xtts_v2")  # downloads to TTS_HOME
+out = "/tmp/xtts_warm.wav"
+model.tts_to_file(text="초기화 워밍업입니다.", speaker=speaker, language="ko", file_path=out)
+print("[setup-xtts] warm-up KR synth OK ->", out, "speaker:", speaker)
+PY
+
+echo "[setup-xtts] Coqui XTTS-v2 venv ready at /opt/xtts (cache /opt/xtts-cache)"

docker/supervisord.conf

@@ -49,22 +49,22 @@ stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
 stderr_logfile_maxbytes=0
 
-[program:melo-worker]
-; Warm MeloTTS Korean voice (speed 1.5) in its own py3.11 venv. The bridge's
-; synthesize() POSTs here; if this is down the bridge falls back to Piper.
-command=/app/docker/run-if-role.sh full,bot /opt/melo/bin/python /app/bridge/melo_worker.py
+[program:xtts-worker]
+; Warm Coqui XTTS-v2 Korean voice (natural female "Ana Florence") in its own
+; py3.11 venv. The bridge's synthesize() POSTs here; if this is down the bridge
+; falls back to Piper (English) only when XTTS_FALLBACK_PIPER=1.
+command=/app/docker/run-if-role.sh full,bot /opt/xtts/bin/python /app/bridge/xtts_worker.py
 directory=/app
-; HF_HOME points at the dedicated, image-baked melo cache (warmed in
-; setup-melo.sh). The brain's whisper_cache volume is mounted over
-; /root/.cache/huggingface, so without this the pre-cached BERT + KR checkpoint
-; would be shadowed and re-downloaded (and would fail if the host is offline).
-; HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE force pure-cache reads: the pinned old
-; transformers/huggingface_hub otherwise retry the network on every load and
-; error out instead of falling back to the (complete) baked cache.
-; MELO_DEVICE inherits from the container env (compose sets it; default cuda)
-; so the worker runs MeloTTS on the GPU. supervisord interpolates %(ENV_x)s
-; from its own environment, which is the container's.
-environment=MELO_LANGUAGE="KR",MELO_SPEED="1.5",MELO_DEVICE="%(ENV_MELO_DEVICE)s",MELO_WORKER_HOST="127.0.0.1",MELO_WORKER_PORT="8770",HF_HOME="/opt/melo-cache",HF_HUB_OFFLINE="1",TRANSFORMERS_OFFLINE="1"
+; TTS_HOME points at the dedicated, image-baked XTTS cache (warmed in
+; setup-xtts.sh). The brain's whisper_cache volume is mounted over
+; /root/.cache, so a dedicated non-volume cache dir avoids the baked model being
+; shadowed and re-downloaded (which would fail if the host is offline).
+; XTTS_DEVICE / XTTS_SPEAKER / XTTS_LANGUAGE inherit from the container env
+; (compose sets them with defaults: cuda / "Ana Florence" / ko). supervisord
+; interpolates %(ENV_x)s from its own environment, which is the container's — so
+; these must always be set in the env (compose guarantees it) or this expansion
+; fails at startup. COQUI_TOS_AGREED accepts the non-commercial XTTS license.
+environment=XTTS_DEVICE="%(ENV_XTTS_DEVICE)s",XTTS_SPEAKER="%(ENV_XTTS_SPEAKER)s",XTTS_LANGUAGE="%(ENV_XTTS_LANGUAGE)s",XTTS_WORKER_HOST="127.0.0.1",XTTS_WORKER_PORT="8771",TTS_HOME="/opt/xtts-cache",COQUI_TOS_AGREED="1"
 priority=280
 autorestart=true
 stdout_logfile=/dev/stdout

docs/llm_contexts.md

@@ -13,6 +13,7 @@ Every distinct LLM call in Jarvis, what feeds it, what consumes it, and how it i
   - Redacted user query
   - Recent dialogue (last 5 minutes), including in-loop tool-call + tool-role messages from prior replies within the active conversation (tool carryover, `DialogueMemory.record_tool_turn` / `get_recent_turns_with_tools` in [src/jarvis/memory/conversation.py](src/jarvis/memory/conversation.py); per-prompt cap via `cfg.tool_carryover_max_turns` / `tool_carryover_per_entry_chars`; storage cap `_tool_turns_max_storage = 16`; cleared on `stop` signal AND on new-conversation entry; UNTRUSTED WEB EXTRACT fence markers preserved on truncation; both `content` and `tool_calls[*].function.arguments` scrubbed on write)
   - Unified system prompt from [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) + ASR note + tool-protocol guidance. Reply language is resolved by `reply_language_directive(lang, cfg.tts_engine)` where `lang = _resolve_output_language()` — the single source of truth that prefers the settings-web UI value (config JSON `output_language`) over the compose `OUTPUT_LANGUAGE` env, so changing the language in the settings page takes effect. An explicit lock wins (forces "reply only in `<language>`", also forbidding other scripts so small models stop leaking trailing CJK/Hanja); else a Piper/Chatterbox TTS forces English (English-only voices); else (multilingual TTS, no lock) the assistant replies in the user's own language. The directive is inserted near the FRONT of the guidance list so a small model gives it primacy, and the SAME resolved `lang` feeds `build_system_prompt()`, which rewrites the persona's "in the user's language" clause to the locked language so the persona cannot contradict the directive (previously the persona read the raw env while the directive read the config value, so a settings-UI change was honoured by one and ignored by the other). Gated in `_build_initial_system_message()` at [engine.py](src/jarvis/reply/engine.py).
+  - **Operator instructions** (two sources, both framed "Additional instructions from the operator:" and appended near the end of the guidance list): the settings-UI `llm_instructions` config field, and every `*.md` file in `AGENTS_DIR` (default `/app/agents`, bind-mounted from `./agents`). The file-based set is read once per turn by `load_agent_instructions()` in [src/jarvis/system_prompt.py](src/jarvis/system_prompt.py) and concatenated in filename order, so dropping/editing a `.md` applies on the next reply with no rebuild/restart; fail-open to `""` when the folder is absent/empty/unreadable.
   - **Warm profile block** (query-agnostic User + Directives excerpt from the knowledge graph, composed by `build_warm_profile()` / `format_warm_profile_block()` in [src/jarvis/memory/graph_ops.py](src/jarvis/memory/graph_ops.py) at Step 3.5 of `reply()`; no LLM call, pure SQLite read; injected unconditionally so personalisation is the default; result cached in `DialogueMemory._hot_cache` under `DialogueMemory.WARM_PROFILE_CACHE_KEY` for the lifetime of the active conversation. Invalidated on `stop`, on new-conversation entry, AND on User/Directives graph mutations via the listener registered in [src/jarvis/daemon.py](src/jarvis/daemon.py) against `register_graph_mutation_listener` in [src/jarvis/memory/graph.py](src/jarvis/memory/graph.py); World-branch writes are ignored)
   - Digested memory enrichment (optional, see #4)
   - Time + location context (re-injected each turn)

src/jarvis/reply/engine.py

@@ -9,7 +9,11 @@ import os
 from typing import Optional, TYPE_CHECKING
 
 from ..utils.redact import redact
-from ..system_prompt import build_system_prompt, reply_language_directive
+from ..system_prompt import (
+    build_system_prompt,
+    load_agent_instructions,
+    reply_language_directive,
+)
 from ..tools.registry import run_tool_with_retries, generate_tools_description, generate_tools_json_schema, BUILTIN_TOOLS
 from ..tools.builtin.stop import STOP_SIGNAL
 from ..debug import debug_log
@@ -1702,6 +1706,10 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
     # the directive used the config value made the two contradict each other.
     _output_language = _resolve_output_language()
     _persona_prompt = build_system_prompt(_assistant_name, _output_language)
+    # File-based operator instructions: every *.md in AGENTS_DIR (default
+    # /app/agents, bind-mounted from ./agents). Read once per turn so edits in
+    # the folder apply on the next reply without a restart; fail-open to "".
+    _agent_instructions = load_agent_instructions()
 
     def _build_initial_system_message() -> str:
         guidance = [_persona_prompt.strip()]
@@ -1810,6 +1818,12 @@ def run_reply_engine(db: "Database", cfg, tts: Optional[Any],
         if _user_instructions:
             guidance.append("Additional instructions from the operator:\n" + _user_instructions)
 
+        # File-based operator instructions: the concatenated agents/*.md content
+        # resolved once above. Same framing/placement as the settings-UI field
+        # so both are treated as authoritative operator guidance.
+        if _agent_instructions:
+            guidance.append("Additional instructions from the operator:\n" + _agent_instructions)
+
         # Recency reinforcement: repeat the language lock at the very END too.
         # In a ~5k-token prompt the front-placed rule gets "lost in the middle";
         # bigger models (qwen2.5:7b) otherwise leak Chinese/Cyrillic mid-reply.

src/jarvis/system_prompt.py

@@ -6,8 +6,51 @@ who renames the wake word (e.g. "Friday") gets a butler with the matching
 name rather than a persona hardcoded to "Jarvis".
 """
 
+import os
+from pathlib import Path
 from typing import Optional
 
+# Default location of the operator's file-based instruction folder. In the
+# Docker deployment ./agents is bind-mounted here (see docker-compose.yml), so a
+# user can drop *.md files in without rebuilding. Overridable via AGENTS_DIR.
+_DEFAULT_AGENTS_DIR = "/app/agents"
+
+
+def load_agent_instructions(agents_dir: Optional[str] = None) -> str:
+    """Concatenate every ``*.md`` in the agents dir into one instruction block.
+
+    Files are read in filename order (so ``00-tone.md`` precedes ``10-rules.md``)
+    and joined with blank lines. This lets the operator extend the main reply
+    LLM's system prompt by dropping Markdown files into a folder, no code change
+    or restart required — the caller reads this once per turn.
+
+    Resolution order for the directory: explicit ``agents_dir`` arg, then the
+    ``AGENTS_DIR`` env var, then ``/app/agents``.
+
+    Fail-open by design: a missing or empty directory, an unreadable file, or
+    any unexpected error yields ``""`` so a misconfigured folder can never break
+    a reply. Only regular ``*.md`` files are read; other files are ignored.
+    """
+    directory = agents_dir or os.environ.get("AGENTS_DIR") or _DEFAULT_AGENTS_DIR
+    try:
+        base = Path(directory)
+        if not base.is_dir():
+            return ""
+        parts: list[str] = []
+        for path in sorted(base.glob("*.md"), key=lambda p: p.name):
+            if not path.is_file():
+                continue
+            try:
+                text = path.read_text(encoding="utf-8").strip()
+            except Exception:
+                continue
+            if text:
+                parts.append(text)
+        return "\n\n".join(parts).strip()
+    except Exception:
+        return ""
+
+
 _SYSTEM_PROMPT_TEMPLATE: str = (
     "Persona: you are a British butler named {name} — polite, composed, quietly amused, and "
     "quietly enjoying yourself. Default voice is dry, witty, and lightly sarcastic: you notice "

tests/test_system_prompt.py

@@ -7,6 +7,7 @@ hardcoded to Jarvis.
 
 from jarvis.system_prompt import (
     build_system_prompt,
+    load_agent_instructions,
     output_language_directive,
     reply_language_directive,
     ENGLISH_ONLY_DIRECTIVE,
@@ -100,6 +101,14 @@ class TestReplyLanguageDirective:
         # user's own language, so no directive.
         assert reply_language_directive(None, "melo") is None
 
+    def test_xtts_is_multilingual(self):
+        # XTTS-v2 (the Korean voice) is not English-only: no lock → free, and a
+        # lock is honoured (not overridden to English).
+        assert reply_language_directive(None, "xtts") is None
+        directive = reply_language_directive("Korean", "xtts")
+        assert directive is not None and "Korean" in directive
+        assert directive != ENGLISH_ONLY_DIRECTIVE
+
     def test_unknown_tts_defaults_to_english_only(self):
         # Preserves the original getattr(cfg, 'tts_engine', 'piper') default:
         # an unknown/missing engine is treated conservatively as English-only.
@@ -108,3 +117,57 @@ class TestReplyLanguageDirective:
     def test_lock_wins_even_with_multilingual_tts(self):
         directive = reply_language_directive("Korean", "melo")
         assert directive is not None and "Korean" in directive
+
+
+class TestLoadAgentInstructions:
+    """Operator can extend the reply LLM's system prompt by dropping *.md files
+    into an agents/ folder. The loader concatenates them in filename order and
+    fails open so a missing/empty/broken folder never breaks a reply."""
+
+    def test_missing_dir_returns_empty(self, tmp_path):
+        assert load_agent_instructions(str(tmp_path / "does-not-exist")) == ""
+
+    def test_empty_dir_returns_empty(self, tmp_path):
+        assert load_agent_instructions(str(tmp_path)) == ""
+
+    def test_reads_and_concatenates_single_file(self, tmp_path):
+        (tmp_path / "rules.md").write_text("Always be brief.", encoding="utf-8")
+        assert load_agent_instructions(str(tmp_path)) == "Always be brief."
+
+    def test_files_are_ordered_by_filename(self, tmp_path):
+        # Filename prefixes let the operator control ordering.
+        (tmp_path / "10-second.md").write_text("SECOND", encoding="utf-8")
+        (tmp_path / "00-first.md").write_text("FIRST", encoding="utf-8")
+        result = load_agent_instructions(str(tmp_path))
+        assert result.index("FIRST") < result.index("SECOND")
+
+    def test_only_md_files_are_read(self, tmp_path):
+        (tmp_path / "note.txt").write_text("IGNORE ME", encoding="utf-8")
+        (tmp_path / "use.md").write_text("USE ME", encoding="utf-8")
+        result = load_agent_instructions(str(tmp_path))
+        assert "USE ME" in result
+        assert "IGNORE ME" not in result
+
+    def test_blank_files_are_skipped(self, tmp_path):
+        (tmp_path / "blank.md").write_text("   \n  ", encoding="utf-8")
+        (tmp_path / "real.md").write_text("Real instruction.", encoding="utf-8")
+        assert load_agent_instructions(str(tmp_path)) == "Real instruction."
+
+    def test_env_var_is_used_when_no_arg(self, tmp_path, monkeypatch):
+        (tmp_path / "a.md").write_text("FROM ENV", encoding="utf-8")
+        monkeypatch.setenv("AGENTS_DIR", str(tmp_path))
+        assert load_agent_instructions() == "FROM ENV"
+
+    def test_explicit_arg_overrides_env(self, tmp_path, monkeypatch):
+        (tmp_path / "env.md").write_text("ENV", encoding="utf-8")
+        other = tmp_path / "other"
+        other.mkdir()
+        (other / "arg.md").write_text("ARG", encoding="utf-8")
+        monkeypatch.setenv("AGENTS_DIR", str(tmp_path))
+        assert load_agent_instructions(str(other)) == "ARG"
+
+    def test_a_file_path_instead_of_dir_returns_empty(self, tmp_path):
+        f = tmp_path / "file.md"
+        f.write_text("x", encoding="utf-8")
+        # Pointed at a file, not a directory → fail-open.
+        assert load_agent_instructions(str(f)) == ""