The streaming splitter treated every "." as a sentence boundary, so the operational reply "17.5°C" was read as "17." / "5°C" and "1.8 km/h" as "1." / "8 km/h" - numbers spoken digit-by-digit plus extra TTS calls. An ASCII terminator (. ! ?) now only ends a sentence when it is followed by whitespace, a closing quote/bracket, or end of text. In-token dots (decimals "17.5", versions "v2.0", hosts "example.com") are followed by a digit/letter, so they no longer split. CJK fullwidth terminators stay unconditional since those scripts use no trailing space. Language-agnostic, punctuation only. - bridge: lookahead-gated boundary regex + finditer-based chunking - tests: regression cases for decimals (17.5/1.8), versions, URLs, and an integer that genuinely ends a sentence Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
4.4 KiB
4.4 KiB