MeloTTS's single Korean speaker sounded non-native ("foreign accent"). Swap it
for Coqui XTTS-v2 with the built-in female studio speaker "Ana Florence"
(language ko), the natural voice used in earlier local runs.
- bridge/xtts_worker.py: new warm HTTP worker (own /opt/xtts venv), same
/synth + /health contract and PCM16 output as the old melo worker
- docker/setup-xtts.sh: builds the venv with cu128 torch (Blackwell) + Coqui
TTS and bakes the XTTS-v2 model offline. Pins transformers>=4.57,<5 (5.x
removed isin_mps_friendly, breaking XTTS) and installs the [codec] extra
(torch>=2.9 needs torchcodec) — both verified by a real host synth
- Dockerfile: replace the melo build layer with the xtts layer
- supervisord.conf: melo-worker -> xtts-worker, env passthrough for
XTTS_DEVICE/SPEAKER/LANGUAGE (always set via compose defaults)
- bridge/server.py: default TTS_ENGINE=xtts, route to the xtts worker, generic
worker-synth helper, neural-only fallback flag (XTTS_FALLBACK_PIPER)
- settings UI: engine dropdown xtts/piper, drop the dead melo_speed field, fix
the supervisorctl restart target to xtts-worker
- compose/.env.example/README: XTTS_* vars, speaker/language knobs, remove melo
- remove bridge/melo_worker.py and docker/setup-melo.sh
- tests: xtts treated as multilingual (not English-only)
Verified on host: coqui-tts loads XTTS-v2 and synthesises Korean as
"Ana Florence" to a 16-bit mono 24kHz WAV.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Go-Live broadcast encodes with h264_nvenc, but the image only requested
NVIDIA_DRIVER_CAPABILITIES=compute,utility. The NVIDIA Container Toolkit gates
which driver libraries it injects by capability, and the NVENC/NVDEC libs
(libnvidia-encode.so.1 / libnvidia-decode.so.1) come with the `video`
capability. Without it the broadcast ffmpeg dies with
"Cannot load libnvidia-encode.so.1", the capture produces no packets, and
Go-Live never connects, while CUDA workloads (ollama/whisper/melo) and
nvidia-smi keep working because compute+utility are present.
Add `video` so hardware encode is available. Applies to both Linux (CDI) and
Windows Docker Desktop (WSL2).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
.gitattributes pins *.sh to LF, but that only helps after a full working-tree
renormalise, which a Windows build box may not have done. The image build kept
failing at `RUN bash setup-melo.sh` because the checked-out file still had CRLF,
so bash read line 18 as `set -euxo pipefail\r` and aborted with
"set: pipefail: invalid option name".
Strip CR from setup-melo.sh before running it, and normalise all docker/scripts
shell scripts to LF after the app COPY so their shebangs (entrypoint, run-*.sh)
also survive a CRLF checkout. Makes the build EOL-agnostic regardless of host
autocrlf settings.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Install xdotool + wmctrl (late Docker layer, preserves the melo cache) so
the on-screen Chrome gets real X input (visible cursor, char-by-char typing)
instead of synthetic events; falls back to the Playwright API if absent.
- Fix active-tab detection (probe document.visibilityState instead of assuming
tab 0) so sequential ops target the right tab.
- Add back / forward / refresh; new/switch/close tabs via real keyboard
(Ctrl+T / Ctrl+<n> / Ctrl+W) when xdotool is present.
- Auto-dismiss native JS dialogs; closePopups clears blank/popup tabs.
- Report broadcast (Go-Live) state in status from the turn context.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a dedicated MeloTTS Korean voice (speed 1.5) as the primary TTS engine,
served by a long-lived in-container worker so each Discord turn pays only
inference cost, not model-load cost.
- bridge/melo_worker.py: tiny HTTP service in its own /opt/melo py3.11 venv,
keeps the KR model warm, returns PCM16 WAV on POST /synth.
- bridge/server.py: synthesize() routes to the melo worker first; Piper stays
as an opt-in fallback (MELO_FALLBACK_PIPER, default off so Korean is never
mangled through the English voice). /health reports tts_engine.
- docker/setup-melo.sh: builds the isolated venv (pinned torch 2.12.0 /
torchaudio 2.11.0 CPU, MeloTTS pinned to a commit for reproducible rebuilds),
pre-fetches mecab-ko, and warms a dedicated HF cache (/opt/melo-cache) with a
real KR synth so all BERT + KR checkpoint assets are baked into the image.
- docker/supervisord.conf: runs melo-worker before the bridge with
HF_HOME=/opt/melo-cache (the whisper_cache volume shadows the default HF
cache) plus HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE so it reads the baked cache
and never retries the network on load.
- Dockerfile/.env.example: wire the melo build layer and config knobs.
Verified: offline synth passes with --network none and the prod volume mounted;
prod container recreated, all supervisord services up, bot logged in, and an
end-to-end /tts call returns a 44.1kHz mono PCM16 WAV.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses review findings on the dockerized stack:
- Container Chrome search was dead: add --remote-debugging-port + a non-default
--user-data-dir (Chrome 136+ refuses CDP on the default profile), add the
playwright dep (browse-search.mjs connectOverCDP) with browser download
skipped, and connect to 127.0.0.1 not "localhost" (container localhost -> ::1
while Chrome binds IPv4). Verified: browse-search returns real results.
- Broadcast toggle reliability: always offer setBroadcast in screen-share mode
(the embedding/keyword router dropped it for non-English utterances) and make
its description force a tool call. "방송 꺼줘"->stop now 5/5; no false triggers.
- Stop the broadcast on voice leave (no orphaned stream).
- Security: bind VNC/noVNC to loopback by default (VNC_BIND override) and the
bridge to the container loopback (BRIDGE_HOST=127.0.0.1), not published.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Makes the all-in-one image actually run the new real-time-search features and
closes review gaps:
- Gemini OAuth path: install Node 22 + @google/gemini-cli (pinned 0.46.0) in the
image; mount a DEDICATED host dir (~/.config/javis/gemini) holding only the
OAuth creds to /root/.gemini (not the whole ~/.gemini). Verified in-container:
`gemini -p ... -o json` returns a grounded answer with no API key.
- Broadcast audio: add PulseAudio + a headless null-sink (run-pulse.sh, new
supervisor program); export XDG_RUNTIME_DIR/PULSE_SERVER so Chrome playback
and the selfbot `ffmpeg -f pulse -i @DEFAULT_MONITOR@` share one daemon.
Verified: default sink virtual_speaker, monitor present, ffmpeg capture OK.
- Bind the brain bridge to 127.0.0.1 only (internal, unauthenticated API).
- VNC host port is overridable; this server pins VNC_PORT=5902 (.env) since the
host already runs Xvnc on 5901.
Verified in-container with CDI GPU passthrough: RTX 5050 visible, NVENC
encoders (h264/hevc/av1) available.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- voice.ts: reply playback is now a FIFO queue (AudioPlayerStatus.Idle drains
it) so concurrent speakers no longer cut each other's replies off.
- selfbot.ts: rewritten against the REAL @dank074/discord-video-stream v6 API
(verified from its d.ts): prepareStream(input, opts, signal)->{command,output},
playStream(output, streamer, {type:"go-live"}, signal), Streamer.joinVoice.
x11grab via customInputOptions; optional NVENC encode (RTX 5050) via exported
`nvenc`. package.json pinned to ^6.0.0 (was a wrong ^4.2.1).
- Dockerfile: dropped the hardcoded python3.12 LD_LIBRARY_PATH. faster-whisper
>=1.1 self-locates the pip CUDA libs; ldconfig (full path, glob) registers
them as a robust fallback. Verified: ld.so cache lists libcublas/libcudnn and
GPU whisper works with LD_LIBRARY_PATH empty.
- bridge: STT resample 48k->16k upgraded from nearest-neighbor to linear
(np.interp).
Verified: tsc clean, image builds, GPU whisper OK via ldconfig, compose valid.
GPU acceleration is now on by default and verified end-to-end on the
Blackwell RTX 5050 (sm_120):
- Ollama offloads 100% to GPU (log: library=CUDA compute=12.0,
BLACKWELL_NATIVE_FP4=1). compose passes GPU via CDI
(devices: nvidia.com/gpu=all) to both ollama and javis.
- Whisper STT on GPU: faster-whisper>=1.1.0 + nvidia-cublas/cudnn cu12,
LD_LIBRARY_PATH baked into the image. Verified float16 transcribe on
sm_120; bridge auto-falls back to CPU when no GPU is present.
- Model: default chat model -> qwen3:8b (best 8GB-VRAM tool-calling,
~5GB Q4). Embed stays nomic-embed-text.
- README documents the host one-time setup (nvidia-container-toolkit +
`nvidia-ctk cdi generate`) and GPU on/off.
Verified: image builds; GPU visible in both containers via compose;
ollama ps = 100% GPU; faster-whisper cuda OK + CPU fallback OK;
bridge /health 200.
`docker compose up -d --build` now brings up the whole thing automatically —
no host setup needed:
- All-in-one javis image: TigerVNC+XFCE desktop, Chrome, Python brain bridge,
Node/bun bot, managed by supervisord (verified: all 6 programs RUNNING).
- ollama service + one-shot ollama-init that auto-pulls chat+embed models
(verified end-to-end; `ollama list` shows pulled models).
- Discord token deferred: without DISCORD_BOT_TOKEN the desktop, bridge,
Ollama and models all run; only the bot waits (no crash loop).
- Slim container deps (bridge/requirements-bridge.txt) drop the unused
PyQt6/torch/chatterbox/sounddevice stack. Piper voice + Whisper models
auto-download into named volumes.
- Configurable host ports (VNC_PORT/NOVNC_PORT/BRIDGE_PORT) to avoid clashing
with a host VNC already on 5901. Bridge binds 0.0.0.0 in-container.
Verified: image builds; brain imports; bridge /health 200; noVNC 200;
X display :1 @1920x1080; auto-pull completes; supervisorctl status all RUNNING.
