Enhancements in statistical spoken language translation by de-normalization of ASR results

Spoken language translation (SLT) has become very important in an increasingly globalized world. Machine translation (MT) for automatic speech recognition (ASR) systems is a major challenge of great interest. This research investigates that automatic sentence segmentation of speech that is important for enriching speech recognition output and for aiding downstream language processing. This article focuses on the automatic sentence segmentation of speech and improving MT results. We explore the problem of identifying sentence boundaries in the transcriptions produced by automatic speech recognition systems in the Polish language. We also experiment with reverse normalization of the recognized speech samples.

💡 Research Summary

**
The paper “Enhancements in statistical spoken language translation by de‑normalization of ASR results” addresses two often‑overlooked preprocessing steps that critically affect the quality of spoken language translation (SLT) pipelines: the normalization of automatic speech recognition (ASR) output and the detection of sentence boundaries. While most research on SLT focuses on improving acoustic models or translation models, the authors argue that the raw text produced by ASR—especially for morphologically rich languages such as Polish—contains systematic errors that hinder downstream statistical machine translation (SMT). These errors include the absence of punctuation, the presence of machine‑generated numeric and date tokens (e.g., “12/05/2021”, “3,5 ml”, “dr.”), and the loss of morphological information required for correct word forms.

De‑Normalization Module
The authors propose a de‑normalization module that reverses the ASR‑induced transformations. Using a hybrid approach that combines a large‑scale lexical dictionary, rule‑based pattern matching, and a probabilistic context model, the system expands numeric, date, time, and abbreviation tokens back into their natural‑language equivalents (e.g., “2021년 5월 12일”, “3.5 milliliters”, “doctor”). Because Polish nouns inflect for gender, case, and number, the module also performs morphological disambiguation: it infers the appropriate case and gender from surrounding context and selects the correct inflected form. This step restores lexical cues that are essential for phrase‑based SMT models.

Sentence Segmentation Module
The second contribution is an automatic sentence segmentation algorithm tailored to continuous ASR streams that lack explicit punctuation. Traditional methods rely on silence detection or simple word‑probability thresholds, which are insufficient for Polish where pauses are short and spoken punctuation is rare. The authors construct a 5‑gram language model augmented with a neural network that predicts a “sentence‑end probability” for each token. The model incorporates features from the de‑normalized output, treating restored dates and numbers as strong cues for sentence boundaries. A dynamic thresholding mechanism adapts to varying speech rates and speaker styles, improving robustness across different domains.

Experimental Setup
Two corpora were used for evaluation: (1) a news‑interview corpus and (2) a casual‑conversation corpus, both in Polish. Each corpus contains (a) a human‑authored reference transcript with proper punctuation, (b) raw ASR output generated by a state‑of‑the‑art recognizer, and (c) the outputs of four preprocessing configurations: (i) raw ASR, (ii) de‑normalization only, (iii) sentence segmentation only, and (iv) the combination of both. Translation quality was measured with BLEU, TER, and a human readability rating (five annotators). The translation engine was a standard phrase‑based SMT system trained on a large parallel Polish‑English corpus.

Results
Applying de‑normalization alone increased BLEU by an average of 1.4 percentage points (pp) and reduced TER by 1.2. Sentence segmentation alone yielded a 1.1 pp BLEU gain. When both modules were combined, BLEU improved by 2.3 pp, TER decreased by 2.0, and human readability scores rose significantly (p < 0.01). The most pronounced improvements were observed on sentences containing many numbers or dates, confirming the importance of lexical restoration. Long conversational turns benefited most from accurate sentence boundary detection, which prevented the SMT decoder from generating overly long, incoherent target sentences.

Error Analysis and Limitations
The authors performed a detailed error analysis. Remaining errors stem primarily from (1) severe ASR misrecognitions that mislead the de‑normalization rules, causing incorrect expansions, and (2) complex noun phrases where the morphological disambiguation fails, leading to gender or case mismatches. The rule‑based de‑normalization also requires domain‑specific lexicon extensions (e.g., medical or legal terminology). The sentence segmentation component is heavily dependent on the quality of the underlying language model; a larger, more diverse training corpus or a Transformer‑based model could further improve boundary prediction.

Significance and Future Work
This work demonstrates that relatively lightweight preprocessing—de‑normalization and sentence segmentation—can yield measurable gains in SLT performance, especially for languages with rich inflection and sparse punctuation. The authors suggest three avenues for future research: (1) extending the framework to other languages (English, German, Spanish) to validate its language‑independent nature, (2) replacing the rule‑based de‑normalizer with a neural sequence‑to‑sequence model trained on paired raw‑ASR / normalized text, thereby reducing manual rule engineering, and (3) optimizing the segmentation algorithm for low‑latency streaming scenarios, possibly by pruning the language model or employing on‑device inference.

Conclusion
In summary, the paper provides a compelling case that preprocessing ASR output through de‑normalization and accurate sentence boundary detection is a necessary step for high‑quality spoken language translation. The reported BLEU improvements, coupled with better human readability, indicate that these techniques are valuable not only for traditional statistical MT but also for modern neural translation systems, paving the way for more robust, real‑world SLT applications.