feat(voice): express [감정] tags via pitch/speed instead of speaking them

Emotion tags now steer delivery rather than being read aloud. parse_segments()
splits a reply on [감정] tags: a recognised emotion word switches the pitch and
speed of the text that follows (and is dropped), while a non-emotion bracket
(e.g. [1번]) keeps its inner words as spoken content. Emotions can change
mid-reply, so a single turn is synthesised as several pitch-shifted segments and
concatenated in the melo worker (librosa pitch_shift, warmed at startup).

The emotion vocabulary is grounded in Azure Neural TTS speaking styles plus
Ekman's basic emotions, with Korean synonyms. The brain persona is updated to
emit inline tags from that set.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
EJClaw
2026-08-22 10:14:49 +09:00
parent f585ed7b76
commit 4db73bf69f
5 changed files with 286 additions and 6 deletions

View File

@@ -28,6 +28,7 @@ from pathlib import Path
from typing import Awaitable, Callable
from ..interfaces import Reply
from .emotion import parse_segments
log = logging.getLogger("wsai.tts.melo")
@@ -173,9 +174,22 @@ class MeloTTS:
callers that want the wav directly (e.g. the Discord voice bridge)."""
await self._ensure()
text = normalize_for_speech(text)
# Split on [감정] tags: each tag steers pitch/speed for the text that
# follows (and is itself not spoken); non-emotion brackets stay as words.
segments = parse_segments(text, self.speed)
self._n += 1
out = str(self.out_dir / f"tts-{self._n:06d}.wav")
req = json.dumps({"text": text, "out": out, "speed": self.speed})
if segments:
payload = {
"segments": [
{"text": s.text, "speed": s.speed, "pitch": s.pitch}
for s in segments
],
"out": out,
}
else: # empty/whitespace reply: keep legacy single-utterance behaviour
payload = {"text": text, "out": out, "speed": self.speed}
req = json.dumps(payload)
s = time.monotonic()
async with self._lock:
assert self._proc and self._proc.stdin and self._proc.stdout