fix(voice): raise the consonant band by 12 dB before transcription

A dark microphone or a speaker who swallows first consonants and
sentence ends loses the band above 1 kHz first: in the live
recordings "자비스" came out as "아비스"/"다비스" and "서울" as "어우".
loudness_equalise now raises it with a one-octave shelf at 1 kHz.

Measured with large-v3-turbo: 25 sure live utterances 5.0% -> 3.4% CER,
FLEURS Korean test split (382 clips, held out) 5.16% -> 4.98%. The real
noise check stays at 108 clips with no text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-11-18 18:09:05 +09:00
parent e428aad58a
commit 9f62bfb059
4 changed files with 34 additions and 3 deletions

View File

@@ -54,6 +54,12 @@ TARGET_DB = -20.0 # loudness every speaker is brought to, dBFS RMS over the
MAX_GAIN_DB = 40.0 # a whisper at -60 dBFS still reaches the target; pure hiss is not blown up
MIN_GAIN_DB = -20.0
HIGHPASS_HZ = 60.0 # mains hum and desk rumble carry no speech
# Consonants ("ㅈ", "ㅅ") live above 1 kHz and are what a dark microphone or a swallowed syllable loses.
# Raised by a shelf that rises over one octave around HIGH_SHELF_HZ. Measured with large-v3-turbo:
# FLEURS Korean (held-out test split, 382 clips) 5.16% -> 4.98% CER, a dark real speaker 5.0% -> 3.4%;
# 6 dB did little, 18-24 dB no better on FLEURS and worse on the real speaker.
HIGH_SHELF_HZ = 1000.0
HIGH_SHELF_DB = 12.0
_KNEE = 0.8 # soft limiter: peaks above this bend smoothly towards the ceiling instead of clipping
_CEILING = 0.98
_WINDOW = 512
@@ -70,7 +76,10 @@ def loudness_equalise(x: np.ndarray, voiced: np.ndarray | None = None) -> np.nda
if len(x) == 0:
return x
spec = np.fft.rfft(x.astype(np.float64))
spec[np.fft.rfftfreq(len(x), 1 / TARGET_RATE) < HIGHPASS_HZ] = 0
freqs = np.fft.rfftfreq(len(x), 1 / TARGET_RATE)
spec[freqs < HIGHPASS_HZ] = 0
rise = np.clip(np.log2(np.maximum(freqs, 1.0) / HIGH_SHELF_HZ) + 0.5, 0.0, 1.0)
spec *= 10 ** (HIGH_SHELF_DB * rise / 20)
y = np.fft.irfft(spec, len(x))
n_win = -(-len(y) // _WINDOW)
frames = np.pad(y, (0, n_win * _WINDOW - len(y))).reshape(n_win, _WINDOW)

View File

@@ -108,6 +108,15 @@ its own streaming Silero VAD (512-sample windows, state kept between frames).
the gain kept between -20 and +40 dB, and peaks above 0.8 are soft-limited below 0.98 instead of
clipping. Measured: speech from -60 dBFS to clipped shouting was transcribed equally well, and quiet
speech under loud clicks or knocks better than with peak normalisation.
Before the level is measured, the band where consonants such as "ㅈ" and "ㅅ" live is raised by
12 dB: a shelf that rises over one octave from 707 Hz to 1.41 kHz (centred on 1 kHz) and stays
flat above. A dark microphone, or a speaker who swallows first consonants and sentence ends, loses
that band first ("자비스" heard as "아비스", "서울" as "어우"). Measured with large-v3-turbo on 32
real Discord utterances of such a speaker: 5.0% character errors on the 25 whose words are sure, down to 3.4%,
with sentence ends ("어때", "틀어줘") and wake words kept that were lost before; on FLEURS
Korean, tuned on the dev split (224 clips, 4.66% to 4.32%) and checked on the held-out test split
(382 clips, 5.16% to 4.98%). 6 dB changed little; 18 and 24 dB were no better on FLEURS and worse on
the real speaker. The noise check below is unchanged with it (108 clips, none with text).
### Scheduling