fix(voice): raise the consonant band by 12 dB before transcription
A dark microphone or a speaker who swallows first consonants and sentence ends loses the band above 1 kHz first: in the live recordings "자비스" came out as "아비스"/"다비스" and "서울" as "어우". loudness_equalise now raises it with a one-octave shelf at 1 kHz. Measured with large-v3-turbo: 25 sure live utterances 5.0% -> 3.4% CER, FLEURS Korean test split (382 clips, held out) 5.16% -> 4.98%. The real noise check stays at 108 clips with no text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -54,6 +54,12 @@ TARGET_DB = -20.0 # loudness every speaker is brought to, dBFS RMS over the
|
||||
MAX_GAIN_DB = 40.0 # a whisper at -60 dBFS still reaches the target; pure hiss is not blown up
|
||||
MIN_GAIN_DB = -20.0
|
||||
HIGHPASS_HZ = 60.0 # mains hum and desk rumble carry no speech
|
||||
# Consonants ("ㅈ", "ㅅ") live above 1 kHz and are what a dark microphone or a swallowed syllable loses.
|
||||
# Raised by a shelf that rises over one octave around HIGH_SHELF_HZ. Measured with large-v3-turbo:
|
||||
# FLEURS Korean (held-out test split, 382 clips) 5.16% -> 4.98% CER, a dark real speaker 5.0% -> 3.4%;
|
||||
# 6 dB did little, 18-24 dB no better on FLEURS and worse on the real speaker.
|
||||
HIGH_SHELF_HZ = 1000.0
|
||||
HIGH_SHELF_DB = 12.0
|
||||
_KNEE = 0.8 # soft limiter: peaks above this bend smoothly towards the ceiling instead of clipping
|
||||
_CEILING = 0.98
|
||||
_WINDOW = 512
|
||||
@@ -70,7 +76,10 @@ def loudness_equalise(x: np.ndarray, voiced: np.ndarray | None = None) -> np.nda
|
||||
if len(x) == 0:
|
||||
return x
|
||||
spec = np.fft.rfft(x.astype(np.float64))
|
||||
spec[np.fft.rfftfreq(len(x), 1 / TARGET_RATE) < HIGHPASS_HZ] = 0
|
||||
freqs = np.fft.rfftfreq(len(x), 1 / TARGET_RATE)
|
||||
spec[freqs < HIGHPASS_HZ] = 0
|
||||
rise = np.clip(np.log2(np.maximum(freqs, 1.0) / HIGH_SHELF_HZ) + 0.5, 0.0, 1.0)
|
||||
spec *= 10 ** (HIGH_SHELF_DB * rise / 20)
|
||||
y = np.fft.irfft(spec, len(x))
|
||||
n_win = -(-len(y) // _WINDOW)
|
||||
frames = np.pad(y, (0, n_win * _WINDOW - len(y))).reshape(n_win, _WINDOW)
|
||||
|
||||
@@ -108,6 +108,15 @@ its own streaming Silero VAD (512-sample windows, state kept between frames).
|
||||
the gain kept between -20 and +40 dB, and peaks above 0.8 are soft-limited below 0.98 instead of
|
||||
clipping. Measured: speech from -60 dBFS to clipped shouting was transcribed equally well, and quiet
|
||||
speech under loud clicks or knocks better than with peak normalisation.
|
||||
Before the level is measured, the band where consonants such as "ㅈ" and "ㅅ" live is raised by
|
||||
12 dB: a shelf that rises over one octave from 707 Hz to 1.41 kHz (centred on 1 kHz) and stays
|
||||
flat above. A dark microphone, or a speaker who swallows first consonants and sentence ends, loses
|
||||
that band first ("자비스" heard as "아비스", "서울" as "어우"). Measured with large-v3-turbo on 32
|
||||
real Discord utterances of such a speaker: 5.0% character errors on the 25 whose words are sure, down to 3.4%,
|
||||
with sentence ends ("어때", "틀어줘") and wake words kept that were lost before; on FLEURS
|
||||
Korean, tuned on the dev split (224 clips, 4.66% to 4.32%) and checked on the held-out test split
|
||||
(382 clips, 5.16% to 4.98%). 6 dB changed little; 18 and 24 dB were no better on FLEURS and worse on
|
||||
the real speaker. The noise check below is unchanged with it (108 clips, none with text).
|
||||
|
||||
### Scheduling
|
||||
|
||||
|
||||
Reference in New Issue
Block a user