feat(tts): add MeloTTS Korean voice via warm worker with offline-baked cache
Adds a dedicated MeloTTS Korean voice (speed 1.5) as the primary TTS engine, served by a long-lived in-container worker so each Discord turn pays only inference cost, not model-load cost. - bridge/melo_worker.py: tiny HTTP service in its own /opt/melo py3.11 venv, keeps the KR model warm, returns PCM16 WAV on POST /synth. - bridge/server.py: synthesize() routes to the melo worker first; Piper stays as an opt-in fallback (MELO_FALLBACK_PIPER, default off so Korean is never mangled through the English voice). /health reports tts_engine. - docker/setup-melo.sh: builds the isolated venv (pinned torch 2.12.0 / torchaudio 2.11.0 CPU, MeloTTS pinned to a commit for reproducible rebuilds), pre-fetches mecab-ko, and warms a dedicated HF cache (/opt/melo-cache) with a real KR synth so all BERT + KR checkpoint assets are baked into the image. - docker/supervisord.conf: runs melo-worker before the bridge with HF_HOME=/opt/melo-cache (the whisper_cache volume shadows the default HF cache) plus HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE so it reads the baked cache and never retries the network on load. - Dockerfile/.env.example: wire the melo build layer and config knobs. Verified: offline synth passes with --network none and the prod volume mounted; prod container recreated, all supervisord services up, bot logged in, and an end-to-end /tts call returns a 44.1kHz mono PCM16 WAV. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
77
docker/setup-melo.sh
Executable file
77
docker/setup-melo.sh
Executable file
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env bash
|
||||
# ============================================================================
|
||||
# Install a dedicated MeloTTS (Korean voice) venv at /opt/melo.
|
||||
#
|
||||
# Why a SEPARATE venv (not the brain-bridge /opt/venv):
|
||||
# - MeloTTS pins old deps (transformers 4.27.4 / tokenizers 0.13.3 / fugashi)
|
||||
# whose binary wheels exist only for cp311, so we use python3.11 here even
|
||||
# though the image's default interpreter is 3.12.
|
||||
# - It isolates the heavy torch/transformers stack from the slim bridge env,
|
||||
# which pins numpy<2 for faster-whisper.
|
||||
#
|
||||
# torch is pinned to the CPU build: TTS runs on CPU so the GPU stays reserved
|
||||
# for Ollama + Whisper, and we avoid pulling multi-GB CUDA wheels.
|
||||
# ============================================================================
|
||||
set -euxo pipefail
|
||||
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
apt-get update
|
||||
# Build deps for fugashi / mecab-python3 + a system MeCab dict, plus python3.11.
|
||||
apt-get install -y --no-install-recommends \
|
||||
software-properties-common build-essential pkg-config swig \
|
||||
libmecab-dev mecab mecab-ipadic-utf8
|
||||
add-apt-repository -y ppa:deadsnakes/ppa
|
||||
apt-get update
|
||||
apt-get install -y --no-install-recommends python3.11 python3.11-venv python3.11-dev
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
python3.11 -m venv /opt/melo
|
||||
/opt/melo/bin/pip install --no-cache-dir --upgrade pip wheel setuptools
|
||||
|
||||
# CPU-only torch first, so MeloTTS's unpinned `torch` dep is already satisfied
|
||||
# and pip does not pull the CUDA build. Pinned for reproducible rebuilds (these
|
||||
# are the versions the CPU index resolved when this layer was verified).
|
||||
/opt/melo/bin/pip install --no-cache-dir torch==2.12.0 torchaudio==2.11.0 \
|
||||
--index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
# MeloTTS from GitHub. The PyPI sdist is broken (its setup.py reads a
|
||||
# requirements.txt that is not shipped in the sdist), so install from the repo.
|
||||
# Pinned to a commit (not refs/heads/main) so rebuilds are reproducible.
|
||||
/opt/melo/bin/pip install --no-cache-dir \
|
||||
"https://github.com/myshell-ai/MeloTTS/archive/209145371cff8fc3bd60d7be902ea69cbdb7965a.tar.gz"
|
||||
|
||||
# Korean g2p backend. MeloTTS otherwise tries to pip-install this on the first
|
||||
# Korean request, which fails in a network-isolated container at runtime.
|
||||
/opt/melo/bin/pip install --no-cache-dir python-mecab-ko python-mecab-ko-dic
|
||||
|
||||
# Remove the full `unidic` package (its dictionary is never downloaded, only a
|
||||
# stub) so mecab-python3 falls back to the bundled `unidic_lite` dict. Without
|
||||
# this, importing melo's Japanese module fails with a missing-mecabrc error.
|
||||
/opt/melo/bin/pip uninstall -y unidic || true
|
||||
|
||||
# Pre-cache every model asset MeloTTS pulls at runtime, so the worker starts
|
||||
# offline and the first Discord turn pays no download cost. Importing melo.api
|
||||
# fetches the Japanese (tohoku-nlp/bert-base-japanese-v3) and Korean
|
||||
# (kykim/bert-kor-base) BERT tokenizers plus nltk g2p data; loading the KR voice
|
||||
# downloads the OpenVoice KR config+checkpoint, and a real synth pulls the
|
||||
# Korean BERT weights. All of these go through huggingface_hub.
|
||||
#
|
||||
# CRITICAL: at runtime docker-compose mounts the `whisper_cache` named volume
|
||||
# over /root/.cache/huggingface (for faster-whisper). That volume would SHADOW
|
||||
# anything baked into the default HF cache, so we pin the melo worker to a
|
||||
# DEDICATED, non-volume cache dir (/opt/melo-cache) here AND in supervisord, and
|
||||
# warm it once. nltk_data (/root/nltk_data) is not volume-mounted so it stays.
|
||||
export HF_HOME=/opt/melo-cache
|
||||
mkdir -p "$HF_HOME"
|
||||
MELO_LANGUAGE=KR HF_HOME=/opt/melo-cache /opt/melo/bin/python - <<'PY'
|
||||
import tempfile
|
||||
from melo.api import TTS
|
||||
|
||||
model = TTS(language="KR", device="cpu")
|
||||
out = tempfile.mktemp(suffix=".wav")
|
||||
model.tts_to_file("초기화 워밍업입니다.", model.hps.data.spk2id["KR"], out, speed=1.5)
|
||||
print("[setup-melo] warm-up KR synth OK ->", out)
|
||||
PY
|
||||
|
||||
echo "[setup-melo] MeloTTS venv ready at /opt/melo"
|
||||
@@ -49,6 +49,26 @@ stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:melo-worker]
|
||||
; Warm MeloTTS Korean voice (speed 1.5) in its own py3.11 venv. The bridge's
|
||||
; synthesize() POSTs here; if this is down the bridge falls back to Piper.
|
||||
command=/opt/melo/bin/python /app/bridge/melo_worker.py
|
||||
directory=/app
|
||||
; HF_HOME points at the dedicated, image-baked melo cache (warmed in
|
||||
; setup-melo.sh). The brain's whisper_cache volume is mounted over
|
||||
; /root/.cache/huggingface, so without this the pre-cached BERT + KR checkpoint
|
||||
; would be shadowed and re-downloaded (and would fail if the host is offline).
|
||||
; HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE force pure-cache reads: the pinned old
|
||||
; transformers/huggingface_hub otherwise retry the network on every load and
|
||||
; error out instead of falling back to the (complete) baked cache.
|
||||
environment=MELO_LANGUAGE="KR",MELO_SPEED="1.5",MELO_DEVICE="cpu",MELO_WORKER_HOST="127.0.0.1",MELO_WORKER_PORT="8770",HF_HOME="/opt/melo-cache",HF_HUB_OFFLINE="1",TRANSFORMERS_OFFLINE="1"
|
||||
priority=280
|
||||
autorestart=true
|
||||
stdout_logfile=/dev/stdout
|
||||
stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:bridge]
|
||||
command=/opt/venv/bin/python -m bridge.server
|
||||
directory=/app
|
||||
|
||||
Reference in New Issue
Block a user