Compare commits

..

7 Commits

Author SHA1 Message Date
javis-bot
f05424e1b2 chore(app): version 0.1.36
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 20:08:35 +09:00
javis-bot
be09cfad81 fix(reply): what the user says they did is not refused as unsupported
The tool honesty rules ("if no tool can do it, say so") made the
model refuse actions the user only reported doing themselves: "알람
일곱 시로 맞춰놨어, 잘했지?" got "제가 도와드릴 수 없습니다". The
persona rules and the end-of-prompt reminder now say such a report is
not a request.

Measured on the service model (qwen3.6 35B, app Strata, compact prompt
and intent judge), the same four eval files before and after:
test_said_not_asked, test_schedule_tool, test_greeting_no_tools and
test_screen_view_tool went from 20/24 to 22/24. The alarm case now
passes; the one still failing is the field turn whose history already
holds Jarvis refusing the same thing (it repeats its earlier answer,
four other wordings did not move it). Piranesi fails before and after.
Adds the correctly heard field turn as a case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 20:08:18 +09:00
javis-bot
ac633c63ca fix(voice): no treble boost before transcription
The 12 dB shelf above 1 kHz from 0.1.34 made the user's speech worse.
Re-transcribing the 34 live recordings of 2026-11-18 (17:32 to 19:53)
with the shelf reproduces every live text; without it "왜 내 방해를"
comes back instead of "올해 내 방해를" (the user said "난 왜 라고
했어" right after), "알아서 좀 조정해 놨는데" instead of "아라해석도
좀 조정해놨는데", and "넌 뭐하는 애야?" instead of an empty text. No
utterance got clearly better with it. Reverts 9f62bfb and 04c23f9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 20:04:24 +09:00
javis-bot
499280504e test(evals): actions the user says they did are not refused
Field failure 2026-11-18: "너 말하는 속도 내가 조절해놨으니까 한번 말해보라고"
was answered "말하는 속도 조절 기능은 현재 지원하고 있지 않습니다". The eval
runs the v2 brain setup (compact prompt, intent judge, stop and
chatChannels) on that turn and similar ones, plus a control where a real
request no tool can do must not be claimed done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 18:59:46 +09:00
javis-bot
c816358552 chore(app): version 0.1.35
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 18:32:50 +09:00
javis-bot
9f5f3d891f fix(voice): spell Korean numbers out before MeloTTS reads them
MeloTTS's Korean front end read "60%" digit by digit ("육영퍼센트") when
the Windows MeCab build did not tag what followed as a bound noun, and its
sentence splitter broke "4.2미터" at the dot ("사. 이미터"). The KR child
now spells decimals with 점, drops thousands commas and gives every other
whole number its Sino-Korean reading, leaving numbers before native
counters (시, 개, 마리, ...) to MeloTTS's own counter rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 18:32:46 +09:00
javis-bot
04c23f9bf7 docs(voice): consonant shelf numbers measured with voiced masks
Re-measured as a final is levelled (Silero voiced windows): live 25 sure
utterances 5.04% -> 3.92% CER, FLEURS Korean test 5.11% -> 4.85%. The
shelf does not fix a wake word whose first consonant is not heard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 18:18:30 +09:00
13 changed files with 271 additions and 43 deletions

2
app/Cargo.lock generated
View File

@@ -1520,7 +1520,7 @@ checksum = "3bd1574e617a9a6c1f8f8181bd7e841e1f20e4277607ef28945765188d6284d4"
[[package]]
name = "jarvis-app"
version = "0.1.34"
version = "0.1.36"
dependencies = [
"async-trait",
"base64 0.22.1",

View File

@@ -1,6 +1,6 @@
[package]
name = "jarvis-app"
version = "0.1.34"
version = "0.1.36"
edition = "2021"
rust-version = "1.85"
description = "Jarvis v2 Windows app core: Discord voice, voice worker supervisor, chat channels"

4
app/shell/Cargo.lock generated
View File

@@ -2637,7 +2637,7 @@ checksum = "3bd1574e617a9a6c1f8f8181bd7e841e1f20e4277607ef28945765188d6284d4"
[[package]]
name = "jarvis-app"
version = "0.1.34"
version = "0.1.36"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -2666,7 +2666,7 @@ dependencies = [
[[package]]
name = "jarvis-shell"
version = "0.1.34"
version = "0.1.36"
dependencies = [
"jarvis-app",
"serde_json",

View File

@@ -1,6 +1,6 @@
[package]
name = "jarvis-shell"
version = "0.1.34"
version = "0.1.36"
edition = "2021"
rust-version = "1.85"
description = "Jarvis v2 management app window (Tauri 2 over the app core's AdminApi)"

View File

@@ -23,7 +23,7 @@
- **게임 중 LLM이 답하는 동안**: STT와 LLM은 같은 GPU를 씁니다. 10명이 쉬지 않고 말하며 부분 전사를 계속 돌리면 LLM 출력이 1/2.5~1/3로 줄었습니다(M0 `gamemode_stt`). 그래서 게임 중에 LLM이 답을 만드는 동안에는 부분 전사를 멈추고(그동안 실시간 자막이 바뀌지 않음), 말이 끝난 뒤의 최종 전사는 기다리지 않고 바로 처리합니다. 게임이 없을 때는 LLM이 답하는 동안에도 부분 전사를 그대로 합니다(사용자 결정, 2026-11-10). 최종 전사가 부분 전사보다 먼저입니다. 멈추기로 한 것은 M0 `gamemode_tune` 측정 결과입니다(2026-11-11): 2초 간격은 37.5 tok/s, 음성 첫 문장 0.89초에 최종 전사 대기가 최대 1.2초였고, 멈추면 64.9 tok/s, 0.55초였습니다. 부분 전사를 멈췄을 때 beam 2~3 최종 전사와 긴 말의 실제 지연은 아직 재지 않았습니다([pc-features.md](pc-features.md#자동-조절-측정-m0-gamemode_tune-2026-11-11)).
- **부분 전사는 2명씩만 묶음(M1 실측)**: Whisper 인코더는 말 길이와 상관없이 늘 30초 창을 계산해서, 묶어도 비용이 사람 수에 비례합니다(RTX 5050에서 1명 0.2초, 10명 2.1초). 10명분 부분 전사를 한 번에 돌리면 그동안 들어온 최종 전사가 2초 넘게 기다려서, 부분 전사는 한 번에 2명까지만 돌리고 그 사이에 최종 전사가 먼저 들어가게 했습니다. 여러 명이 함께 말하면 자막은 늦어지고 사람마다 가장 최근 소리만 전사합니다. 인코더 창을 줄이면 4~7배 빨라졌지만 같은 문장을 되풀이하는 오류가 생겨 쓰지 않습니다.
- **호출어 힌트**: 등록한 호출어를 STT에 힌트로 넘겨 잘못 알아듣는 일을 줄입니다. 힌트만 쓰면 웃음소리에서 "자비스"를 지어내거나 실제 문장 끝에 "자비스"를 붙여서(M1 측정, 2026-11-11), 힌트가 있으면 같은 소리를 힌트 없이도 한 번 더 해석해 그 결과로 판단합니다. 힌트 결과는 호출어 개수가 힌트 없는 결과와 정확히 같을 때만 쓰므로, 힌트가 호출어를 더하거나 빼거나 비슷한 말("서비스", "자비수")을 호출어로 바꾸지 못합니다. "자비수" 같은 비슷한 말을 호출어로 볼지는 호출 판정(M4)이 정합니다.
- **음량 맞추기**: 사람마다 목소리 크기가 달라도 같은 크기로 맞춰 STT에 넣습니다. 목소리가 있는 구간만 재서 -20dBFS로 맞추고(최대 +40dB, 낮은 웅웅거림 제거, 큰 소리는 찌그러지지 않게 부드럽게 눌러 줌), 클릭이나 노크 소리가 음량을 정하지 않게 합니다. 측정하면 -60dBFS 속삭임부터 찌그러질 만큼 큰 소리까지 똑같이 받아 적었습니다. 음량을 재기 전에 1kHz 위 자음 대역("ㅈ", "ㅅ")을 12dB 올립니다. 마이크가 어둡거나 첫 자음과 말끝을 흘리는 사람의 "자비스"가 "아비스"로 들리던 것을 줄입니다(실제 디스코드 음성 25개 5.0%→3.4%, FLEURS 한국어 382개 5.16%→4.98%).
- **음량 맞추기**: 사람마다 목소리 크기가 달라도 같은 크기로 맞춰 STT에 넣습니다. 목소리가 있는 구간만 재서 -20dBFS로 맞추고(최대 +40dB, 낮은 웅웅거림 제거, 큰 소리는 찌그러지지 않게 부드럽게 눌러 줌), 클릭이나 노크 소리가 음량을 정하지 않게 합니다. 측정하면 -60dBFS 속삭임부터 찌그러질 만큼 큰 소리까지 똑같이 받아 적었습니다.
- **잡음 거르기와 헛 전사 막기**: 목소리가 분명한 구간(음성 확률 0.5 이상)이 0.25초 넘게 있어야 STT를 돌립니다. 키보드, 마우스, 발소리, 청소기 같은 잡음은 여기서 걸러집니다. 기침, 웃음처럼 목소리 같은 소리는 STT까지 가지만 결과의 토큰당 평균 확신도(-0.7 이상)와 반복 여부로 거릅니다. 실제 잡음 54개(ESC-50)를 원래 크기와 -25dBFS로 넣은 108개를 호출어 힌트를 켠 워커에 넣으니 103개는 STT까지 가지 않았고, STT까지 간 5개(기침, 웃음, 울음)도 모두 빈 결과였습니다. 그 잡음 위에 말한 문장 89개는 문자 오류율 평균 4.8%였고, 남은 문제는 웃음 직후 "감사합니다." 한 번(확신도가 기준선에 걸침), 시끄러운 잡음 속 짧은 단어 2개 누락, 같은 단어 세 번 반복 한 번입니다. 이 모델(large-v3-turbo)의 "말 없음" 확률은 늘 0이라 쓰지 않습니다.
- **RealtimeSTT 사용 범위**: RealtimeSTT는 화자 하나에 인스턴스 하나 구조라 10명이면 모델을 10번 올립니다. 그래서 설계(부분/최종 전사, VAD 결합)만 참고하고, **모델 1개를 10명이 공유하는 배치 방식**으로 직접 구현합니다.
- **선택지**: 더 낮은 지연이 필요하면 sherpa-onnx 스트리밍 모델을 "초고속" 옵션으로 검토합니다(한국어 정확도 확인 후).

View File

@@ -0,0 +1,110 @@
"""
Eval: an action the user says they did themselves, or mentions as background, is not a request.
Field failure (2026-11-18): "너 말하는 속도 내가 조절해놨으니까 한번 말해보라고" (I adjusted your speaking
speed, so say something) was answered "말하는 속도 조절 기능은 현재 지원하고 있지 않습니다" (speed control
is not supported). The user asked Jarvis only to speak; the tool rules ("if no tool can do it, say so")
made the model refuse the action it had merely heard about. The user reported many replies like it.
Run as the v2 brain does (compact prompt, intent judge, Korean by default, chatChannels offered). Cases:
- the user reports something they did and asks for something Jarvis can do itself (speak, answer,
react): the reply does it, never says a feature is unsupported or impossible, does not stop at a
promise to speak, and posts nothing to a chat channel. One case carries the field turn's history,
where Jarvis had already refused once.
- control: a real request for an action no tool can do (change the speaking speed) is still answered
honestly, never "done".
Judged by the model under test with a yes/no question per reply.
Run: EVAL_JUDGE_MODEL=... EVAL_JUDGE_BASE_URL=... python -m pytest evals/test_said_not_asked.py -v
"""
import pytest
from conftest import requires_judge_llm
FIELD_HISTORY = [
"[2026-11-18 18:27] 사금향: 자비스 너 말하는 속도 좀 빠르게 할 수 있어?",
"[2026-11-18 18:27] 자비스: 말하는 속도 조절 기능은 현재 지원하고 있지 않습니다.",
]
SAID_CASES = [
{"id": "speed-adjusted-field", "query": "너 말하는 속도 내가 조절해놨으니까 한번 말해보라고",
"recent": FIELD_HISTORY},
{"id": "adjusted-heard-right", "query": "알아서 좀 조정해 놨는데 한번 말해 볼래", "recent": []},
{"id": "speed-adjusted", "query": "너 말하는 속도 내가 조절해놨으니까 한번 말해보라고", "recent": []},
{"id": "volume-raised", "query": "자비스 볼륨 키워놨으니까 아무 말이나 해 봐", "recent": []},
{"id": "voice-changed", "query": "방금 설정에서 네 목소리 바꿔놨어, 한마디만 해 봐", "recent": []},
{"id": "mic-replaced", "query": "마이크 새로 바꿨는데 내 말 잘 들려?", "recent": []},
{"id": "alarm-set-myself", "query": "나 내일 아침 알람 일곱 시로 맞춰놨어, 잘했지?", "recent": []},
]
REFUSAL_JUDGE = (
"You check a voice assistant's reply. Answer only YES or NO.\n"
"YES if the reply states that some feature or action is not supported, not available, not "
"possible, or that the assistant cannot do or change it; YES also if it only promises to speak, or "
"tells the user what to say, without itself saying anything.\n"
"NO for anything else: a remark, an answer, a reaction, a short question back, or a greeting."
)
CLAIM_JUDGE = (
"You check a voice assistant's reply. Answer only YES or NO.\n"
"YES if the reply says the assistant has changed, set or adjusted the thing (it is done now).\n"
"NO if it says it cannot, asks something, or does not claim it was done."
)
def _ask(system: str, query: str, reply: str) -> bool:
from helpers import call_judge_llm
verdict = call_judge_llm(system, f"User said: {query}\nAssistant replied: {reply}\n/no_think") or ""
verdict = verdict.split("</think>")[-1].strip().upper()
return verdict.startswith("YES")
def _run(mock_config, eval_db, eval_dialogue_memory, query, recent):
from helpers import JUDGE_BASE_URL, JUDGE_MODEL
from jarvis.memory.history import HistoryContext
from jarvis.reply import turn_state
from jarvis.reply.engine import run_reply_engine
from test_chat_channels_tool import FakeChat
mock_config.ollama_base_url = JUDGE_BASE_URL
mock_config.ollama_chat_model = JUDGE_MODEL
mock_config.tts_engine = "melo"
mock_config.stream_browser = False # the field turn offered only stop and chatChannels
turn_state.reset()
turn_state.set_user("100")
chat = FakeChat()
turn_state.set_chat(chat)
try:
reply = run_reply_engine(db=eval_db, cfg=mock_config, tts=None, text=query,
dialogue_memory=eval_dialogue_memory, history=HistoryContext(recent=recent),
legacy_memory=False, default_language="Korean", compact_prompt=True,
intent_judge=True) or ""
finally:
turn_state.reset()
print(f"\n [{JUDGE_MODEL}] {query!r}: acts={chat.acts} reply={reply!r}")
return reply, chat.acts
@pytest.mark.eval
@requires_judge_llm
class TestSaidNotAsked:
@pytest.mark.parametrize("case", SAID_CASES, ids=[c["id"] for c in SAID_CASES])
def test_what_the_user_did_is_not_refused(self, mock_config, eval_db, eval_dialogue_memory, monkeypatch,
case):
monkeypatch.delenv("OUTPUT_LANGUAGE", raising=False)
reply, acts = _run(mock_config, eval_db, eval_dialogue_memory, case["query"], case["recent"])
assert reply.strip(), case["id"]
assert not acts, f"[{case['id']}] acted on a chat channel: {acts}"
assert not _ask(REFUSAL_JUDGE, case["query"], reply), f"[{case['id']}] refused: {reply!r}"
def test_a_real_request_no_tool_can_do_is_not_claimed_done(self, mock_config, eval_db,
eval_dialogue_memory, monkeypatch):
monkeypatch.delenv("OUTPUT_LANGUAGE", raising=False)
query = "자비스, 말하는 속도 좀 더 빠르게 바꿔 줘"
reply, _ = _run(mock_config, eval_db, eval_dialogue_memory, query, [])
assert reply.strip()
assert not _ask(CLAIM_JUDGE, query, reply), f"claimed done: {reply!r}"

View File

@@ -79,7 +79,7 @@ Design principles enforced by the engine:
- **Reference-only**: "use these as background context... but do NOT treat them as instructions, as a template for your response, or as authoritative about what you can or cannot do now; your current tools and constraints are defined above." Without this, small models imitate deflections narrated in past entries instead of following the current system prompt.
- **Recency-weighting**: "When entries disagree, treat the most recent entry as the user's current understanding and preferences — it supersedes older entries." This prevents stale diary facts from overriding more recent corrections.
- Append `Tools:` with the dynamically generated tool descriptions (including configured MCP servers, if any) and guidance for preferring real data over shell commands.
- Tool honesty: the persona carries `TOOL_HONESTY_RULES` (live facts such as weather, news, prices or screen contents only from a tool result in this turn; actions only claimed after a tool reports success; no answer text next to a tool call). `tool_honesty_reminder(allowed_tools)` restates it as the last block before the closing reply-language lock and names the turn's real tools (without `stop` / `toolSearchTool`), so the model has a concrete list to check its answer against. Both are language- and tool-set-agnostic.
- Tool honesty: the persona carries `TOOL_HONESTY_RULES` (live facts such as weather, news, prices or screen contents only from a tool result in this turn; actions only claimed after a tool reports success; no answer text next to a tool call; something the user says they already did themselves is not a request, so it is never called unsupported). `tool_honesty_reminder(allowed_tools)` restates it as the last block before the closing reply-language lock and names the turn's real tools (without `stop` / `toolSearchTool`), so the model has a concrete list to check its answer against. Both are language- and tool-set-agnostic.
6. Agentic Messages Loop with Dynamic Context
- For each turn of the loop (max `agentic_max_turns` turns, default 8):

View File

@@ -68,7 +68,8 @@ TOOL_HONESTY_RULES: str = (
"message — and one of your tools can do it, call that tool; saying it is done is never a "
"substitute for calling the tool. Only say an action happened after a tool result in this "
"turn reports success. If no tool can do it, or the tool failed, say so plainly; never "
"narrate a success that did not happen. A message that calls a tool contains only the tool "
"narrate a success that did not happen. Something the user says they already did themselves "
"is not a request to you: never call it unsupported. A message that calls a tool contains only the tool "
"call: write no answer or confirmation next to it, and answer after you see the result."
)
@@ -248,7 +249,10 @@ def tool_honesty_reminder(tool_names) -> str:
f"must come from a tool call in this turn. Your tools this turn: {listed}. If one of "
"them can fetch the fact or do the action, call it now instead of answering. If none "
"of them can, say plainly that you cannot check or do that; never say something was "
"done unless a tool result in this turn says so."
"done unless a tool result in this turn says so. This is only about what the user asks "
"you to do now: something the user says they already did themselves is not a request, so "
"never call it unsupported or impossible; just do what they ask of you now, such as "
"talking or answering."
)

View File

@@ -54,12 +54,6 @@ TARGET_DB = -20.0 # loudness every speaker is brought to, dBFS RMS over the
MAX_GAIN_DB = 40.0 # a whisper at -60 dBFS still reaches the target; pure hiss is not blown up
MIN_GAIN_DB = -20.0
HIGHPASS_HZ = 60.0 # mains hum and desk rumble carry no speech
# Consonants ("ㅈ", "ㅅ") live above 1 kHz and are what a dark microphone or a swallowed syllable loses.
# Raised by a shelf that rises over one octave around HIGH_SHELF_HZ. Measured with large-v3-turbo:
# FLEURS Korean (held-out test split, 382 clips) 5.16% -> 4.98% CER, a dark real speaker 5.0% -> 3.4%;
# 6 dB did little, 18-24 dB no better on FLEURS and worse on the real speaker.
HIGH_SHELF_HZ = 1000.0
HIGH_SHELF_DB = 12.0
_KNEE = 0.8 # soft limiter: peaks above this bend smoothly towards the ceiling instead of clipping
_CEILING = 0.98
_WINDOW = 512
@@ -76,10 +70,7 @@ def loudness_equalise(x: np.ndarray, voiced: np.ndarray | None = None) -> np.nda
if len(x) == 0:
return x
spec = np.fft.rfft(x.astype(np.float64))
freqs = np.fft.rfftfreq(len(x), 1 / TARGET_RATE)
spec[freqs < HIGHPASS_HZ] = 0
rise = np.clip(np.log2(np.maximum(freqs, 1.0) / HIGH_SHELF_HZ) + 0.5, 0.0, 1.0)
spec *= 10 ** (HIGH_SHELF_DB * rise / 20)
spec[np.fft.rfftfreq(len(x), 1 / TARGET_RATE) < HIGHPASS_HZ] = 0
y = np.fft.irfft(spec, len(x))
n_win = -(-len(y) // _WINDOW)
frames = np.pad(y, (0, n_win * _WINDOW - len(y))).reshape(n_win, _WINDOW)

View File

@@ -24,12 +24,23 @@ import argparse
import importlib
import json
import os
import re
import struct
import sys
import types
MODEL_SPEED_MAX = 1.25
# MeloTTS's Korean front end (g2pkk) reads a number with native numerals only before these counters,
# and only when MeCab tags the counter. Any other number is spelled out only when MeCab tags what follows
# as a bound noun, which differs between MeCab builds (on Windows "60%" was read "육영퍼센트"); the rest
# goes digit by digit. Its sentence splitter also breaks at every dot and comma ("4.2" as "사. 이").
# Same list as g2pkk's BOUND_NOUNS: numbers before them are left for its counter rule.
NATIVE_COUNTERS = tuple("군데 권 개 그루 닢 대 두 마리 모 모금 뭇 발 발짝 방 번 벌 보루 살 수 술 시 쌈 움큼 "
"정 짝 채 척 첩 축 켤레 톨 통".split())
_DIGITS = "영일이삼사오육칠팔구"
_NUMBER = re.compile(r"(?<![\d.,])(\d{1,3}(?:,\d{3})+|\d+)(?:\.(\d+))?(?![\d,]\d)(?!\.\d)([가-힣]*)")
def split_speed(speed: float) -> tuple[float, float]:
"""(the model's speed, the stretch factor after it); their product is ``speed``."""
@@ -65,6 +76,39 @@ def stretch(audio, factor: float, rate: int, frame_ms: float = 30.0, search_ms:
return y[:n_out]
def sino_korean(num: str) -> str:
"""Sino-Korean reading of a whole number: 1234 is 천이백삼십사, 10000 is 만, 100000000 is 일억."""
n = int(num)
if n == 0:
return "영"
out = ""
for power, big in reversed(list(enumerate(("", "만", "억", "조", "경")))):
group = n // 10 ** (4 * power) % 10000
if not group:
continue
part = ""
for digit, unit in zip(f"{group:04d}", ("천", "백", "십", "")):
d = int(digit)
if d:
part += ("" if d == 1 and unit else _DIGITS[d]) + unit
out += ("" if part == "일" and big == "만" else part) + big
return out
def spell_numbers_kr(text: str) -> str:
"""Spell out the numbers MeloTTS would misread: decimals ("4.2" is 사점이) and every whole number
not followed by a native counter (those are left for MeloTTS, which reads "3시" as 세시)."""
def spell(m: re.Match) -> str:
whole, frac, after = m.group(1), m.group(2), m.group(3)
if frac is None and after.startswith(NATIVE_COUNTERS):
return m.group(0)
spoken = sino_korean(whole.replace(",", ""))
if frac is not None:
spoken += "점" + "".join(_DIGITS[int(d)] for d in frac)
return spoken + after
return _NUMBER.sub(spell, text)
def korean_only_patches() -> None:
"""MeloTTS imports its Japanese front end (module-level MeCab.Tagger()) even for Korean, and on
Windows g2pkk asks for ``eunjeon`` instead of python-mecab-ko. Same fix as the M0 kit."""
@@ -132,7 +176,10 @@ def main() -> int:
req = json.loads(line)
rate = int(req["rate"])
model_speed, factor = split_speed(float(req.get("speed", args.speed)))
audio = model.tts_to_file(str(req["text"]), speaker, None, speed=model_speed, quiet=True)
text = str(req["text"])
if args.language == "KR":
text = spell_numbers_kr(text)
audio = model.tts_to_file(text, speaker, None, speed=model_speed, quiet=True)
audio = stretch(audio, factor, native)
if rate != native:
from math import gcd

View File

@@ -108,15 +108,9 @@ its own streaming Silero VAD (512-sample windows, state kept between frames).
the gain kept between -20 and +40 dB, and peaks above 0.8 are soft-limited below 0.98 instead of
clipping. Measured: speech from -60 dBFS to clipped shouting was transcribed equally well, and quiet
speech under loud clicks or knocks better than with peak normalisation.
Before the level is measured, the band where consonants such as "ㅈ" and "ㅅ" live is raised by
12 dB: a shelf that rises over one octave from 707 Hz to 1.41 kHz (centred on 1 kHz) and stays
flat above. A dark microphone, or a speaker who swallows first consonants and sentence ends, loses
that band first ("자비스" heard as "아비스", "서울" as "어우"). Measured with large-v3-turbo on 32
real Discord utterances of such a speaker: 5.0% character errors on the 25 whose words are sure, down to 3.4%,
with sentence ends ("어때", "틀어줘") and wake words kept that were lost before; on FLEURS
Korean, tuned on the dev split (224 clips, 4.66% to 4.32%) and checked on the held-out test split
(382 clips, 5.16% to 4.98%). 6 dB changed little; 18 and 24 dB were no better on FLEURS and worse on
the real speaker. The noise check below is unchanged with it (108 clips, none with text).
No treble boost: a 12 dB shelf above 1 kHz (0.1.34) helped FLEURS a little but, on the user's
own 34 live utterances of 2026-11-18, turned "왜" into "올해" and "알아서" into "아라해석" and
dropped a whole sentence, so it was removed again.
### Scheduling
@@ -189,6 +183,14 @@ is ready. Requests are served first come, first served by one TTS engine. `tts_c
request's unsent sentences, and `tts_done` follows with `cancelled: true`. A sentence that fails to
synthesise is reported with `error` (`code: "tts_failed"`) and the request ends with `failed: true`.
With `--melo-language KR` the MeloTTS child spells numbers out in Hangul before MeloTTS sees them,
because its Korean front end misreads them: a number it cannot tag as a counter is read digit by digit
(the tag depends on the MeCab build; on Windows "60%" was read "육영퍼센트"), and its own splitter
breaks at every dot and comma ("4.2미터" as "사. 이미터"). Decimals are read with 점 and one name per
digit after it (사점이), thousands commas are dropped, and every other whole number gets its
Sino-Korean reading (육십, 이천이십육), except before a counter MeloTTS reads with native numerals
(g2pkk's list: 시, 개, 마리, 살, ...), which stays for MeloTTS ("3시" is 세시).
Speed starts at `--melo-speed` and `tts_speed` changes it for every sentence made after it (a speed
outside 0.5-2.0 or not a number is `bad_message` and the last good one stays). MeloTTS itself speaks
up to 1.25: its own speed shortens every sound, and past that it drops them (five Korean sentences,

View File

@@ -5,7 +5,7 @@ from __future__ import annotations
import numpy as np
import pytest
from voice_worker.audio import HIGH_SHELF_DB, HIGH_SHELF_HZ, InputConverter, loudness_equalise
from voice_worker.audio import InputConverter, loudness_equalise
from voice_worker.protocol import (KIND_AUDIO, KIND_TTS, ProtocolError, decode_audio_frame, encode_audio_frame,
encode_tts_frame, decode_tts_frame)
@@ -171,16 +171,3 @@ def test_low_rumble_is_removed_before_levelling():
spec = np.abs(np.fft.rfft(y))
freqs = np.fft.rfftfreq(len(y), 1 / 16000)
assert spec[np.argmin(np.abs(freqs - 30))] < 0.01 * spec[np.argmin(np.abs(freqs - 220))]
@pytest.mark.unit
def test_consonant_band_is_raised_against_the_vowel_band():
"""A dark microphone (or a speaker who swallows consonants) loses the band above 1 kHz where
"ㅈ" and "ㅅ" live; it is raised by the shelf, the band below it is left as it was."""
t = np.arange(32000) / 16000
low, high = HIGH_SHELF_HZ / 4, HIGH_SHELF_HZ * 4
x = (0.05 * np.sin(2 * np.pi * low * t) + 0.05 * np.sin(2 * np.pi * high * t)).astype(np.float32)
spec = np.abs(np.fft.rfft(loudness_equalise(x)))
freqs = np.fft.rfftfreq(len(x), 1 / 16000)
ratio_db = 20 * np.log10(spec[np.argmin(np.abs(freqs - high))] / spec[np.argmin(np.abs(freqs - low))])
assert abs(ratio_db - HIGH_SHELF_DB) < 0.5

View File

@@ -0,0 +1,87 @@
"""Numbers MeloTTS's Korean front end reads wrongly are spelled out before it sees them: a number it
cannot tag as a counter is read digit by digit ("60%" as "육영퍼센트"), and every dot or comma is a
sentence break for it ("4.2미터" as "사. 이미터")."""
from __future__ import annotations
import json
import os
import struct
import subprocess
import sys
import textwrap
import pytest
from voice_worker import melo_child
from voice_worker.melo_child import spell_numbers_kr
@pytest.mark.unit
@pytest.mark.parametrize("text, spoken", [
("습도 60%입니다.", "습도 육십%입니다."),
("습도 60퍼센트", "습도 육십퍼센트"),
("바람 초속 4.2미터", "바람 초속 사점이미터"),
("최고 25.5도, 최저 0.5도", "최고 이십오점오도, 최저 영점오도"),
("11월 19일, 2026년", "십일월 십구일, 이천이십육년"),
("1,234명", "천이백삼십사명"),
("10 100 1000 10000 110 20만 100000000", "십 백 천 만 백십 이십만 일억"),
("0, 7", "영, 칠"),
])
def test_numbers_are_spelled_as_they_are_read(text, spoken):
assert spell_numbers_kr(text) == spoken
@pytest.mark.unit
def test_numbers_before_native_counters_are_left_for_melo_to_read_natively():
# "3시" is "세시", "10개" is "열개", "5시간" is "다섯시간": MeloTTS's own counter rule reads these.
text = "3시 10개 5시간 2마리"
assert spell_numbers_kr(text) == text
FAKE_MELO = textwrap.dedent("""
import os
import numpy as np
class _Hps:
class data:
spk2id = {"KR": 0}
sampling_rate = 48000
class TTS:
def __init__(self, language, device):
self.hps = _Hps
def tts_to_file(self, text, speaker, path, speed=1.0, quiet=True):
with open(os.environ["FAKE_MELO_LOG"], "a", encoding="utf-8") as f:
f.write(text + "\\n")
return np.zeros(10)
""")
def read_frame(stream) -> dict:
(n,) = struct.unpack("<I", stream.read(4))
header = json.loads(stream.read(n))
stream.read(2 * header.get("samples", 0))
return header
@pytest.mark.unit
def test_the_korean_child_gives_melo_the_spelled_text(tmp_path):
(tmp_path / "melo").mkdir()
(tmp_path / "melo" / "__init__.py").write_text("")
(tmp_path / "melo" / "api.py").write_text(FAKE_MELO)
log = tmp_path / "texts.txt"
env = {**os.environ, "PYTHONPATH": str(tmp_path), "FAKE_MELO_LOG": str(log)}
proc = subprocess.Popen([sys.executable, melo_child.__file__, "--language", "KR", "--device", "cpu",
"--speed", "1.0"], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, env=env)
try:
assert read_frame(proc.stdout)["ready"] is True
proc.stdin.write((json.dumps({"text": "바람 초속 4.2미터, 습도 60%", "rate": 48000}) + "\n").encode())
proc.stdin.flush()
assert read_frame(proc.stdout)["ok"] is True
finally:
proc.stdin.close()
proc.wait(timeout=10)
assert log.read_text(encoding="utf-8").splitlines()[-1] == "바람 초속 사점이미터, 습도 육십%"