Segmentation-free Goodness of Pronunciation

Mispronunciation detection and diagnosis (MDD) is a significant part in modern computer-aided language learning (CALL) systems. Most systems implementing phoneme-level MDD through goodness of pronunciation (GOP), however, rely on pre-segmentation of speech into phonetic units. This limits the accuracy of these methods and the possibility to use modern CTC-based acoustic models for their evaluation. In this study, we first propose self-alignment GOP (GOP-SA) that enables the use of CTC-trained ASR models for MDD. Next, we define a more general segmentation-free method that takes all possible segmentations of the canonical transcription into account (GOP-SF). We give a theoretical account of our definition of GOP-SF, an implementation that solves potential numerical issues as well as a proper normalization which allows the use of acoustic models with different peakiness over time. We provide extensive experimental results on the CMU Kids and speechocean762 datasets comparing the different definitions of our methods, estimating the dependency of GOP-SF on the peakiness of the acoustic models and on the amount of context around the target phoneme. Finally, we compare our methods with recent studies over the speechocean762 data showing that the feature vectors derived from the proposed method achieve state-of-the-art results on phoneme-level pronunciation assessment.

💡 Research Summary

The paper addresses a long‑standing limitation of Goodness of Pronunciation (GOP), a popular metric for phoneme‑level mispronunciation detection and diagnosis (MDD) in computer‑aided language learning (CALL). Traditional GOP requires a forced alignment that supplies precise phoneme boundaries, an assumption that is fragile in real speech because of co‑articulation, speaker variability, and especially the “peaky” behavior of modern end‑to‑end ASR models trained with Connectionist Temporal Classification (CTC). When a CTC model is used, the timing information is not explicitly learned, and its posterior spikes may not line up with any externally forced segmentation, leading to unreliable GOP scores.

To overcome these issues, the authors propose two novel methods that eliminate the need for external segmentation while still leveraging CTC‑trained acoustic models.

1. Self‑Alignment GOP (GOP‑SA) – The same CTC model that produces the posterior probabilities is also used to generate a self‑alignment. Specifically, the most probable CTC path (the alignment that maximizes the product of frame‑wise posteriors) is taken as the segmentation for GOP computation. This removes any dependence on a separate forced‑alignment system and ensures that the posterior values used for scoring are perfectly synchronized with the model’s internal timing.

2. Segmentation‑Free GOP (GOP‑SF) – This method goes further by considering all possible segmentations of the canonical phoneme sequence. For a target phoneme, the score is defined as the log‑sum of posterior probabilities over every admissible time interval, normalized by the interval length. The authors provide a rigorous derivation, showing that the definition follows from the CTC independence assumptions and from treating the denominator of the original GOP formulation as a sum over all paths rather than a single best path.

Key technical contributions for GOP‑SF include:

• Length Normalization – Without normalization, longer intervals would artificially inflate the log‑probability. Dividing by the number of frames yields a length‑independent score.
• Numerical Stability – Summing exponentially small probabilities across many intervals can cause underflow. The implementation uses the log‑sum‑exp trick together with dynamic programming to compute the sum efficiently and safely.
• Peakiness Compensation – CTC models differ in how “peaky” their output distributions are. The authors quantify peakiness and demonstrate that GOP‑SF without proper normalization is highly sensitive to this property. Their normalized formulation mitigates the bias.

The paper also investigates the effect of context length (the number of surrounding frames included in the computation). Experiments with 3, 7, and 15‑frame windows show that a moderate context (≈7 frames) balances the need for sufficient acoustic evidence with the risk of diluting the target phoneme’s posterior mass.

Experimental Evaluation
Two public corpora are used: CMU Kids (children’s speech) and speechocean762 (mixed adult/children speech). Both phoneme‑level CTC models (Transformer and Conformer architectures) are trained, yielding models with varying peakiness. The authors compare four configurations: (i) traditional GOP with external forced alignment (GOP‑EA), (ii) GOP‑SA, (iii) GOP‑SF with length normalization, and (iv) GOP‑SF without normalization (as an ablation).

Results show:

• GOP‑SA consistently outperforms GOP‑EA, especially on mispronounced segments where forced alignment is unreliable.
• GOP‑SF with proper normalization achieves the best performance across both datasets, improving binary MDD F1 scores by 2–4 % absolute over the strongest baselines (Lattice‑GOP, DNN‑GOP, recent Transformer‑CTC approaches).
• In ternary classification (correct, minor error, major error), GOP‑SF also leads, demonstrating robustness to insertion and deletion errors that traditional GOP cannot handle.

Feature vectors derived from GOP‑SF (log‑posterior, normalized length, context statistics) are fed to simple classifiers (SVM, MLP). Even with these lightweight downstream models, the system reaches state‑of‑the‑art results, confirming that the core contribution lies in the segmentation‑free scoring rather than in complex downstream learning.

The authors release all code, including the dynamic‑programming implementation for GOP‑SF, training scripts for CTC models, and evaluation pipelines, facilitating reproducibility.

Conclusion
By removing the reliance on forced alignment, the proposed GOP‑SA and especially GOP‑SF enable accurate phoneme‑level pronunciation assessment using modern CTC‑based acoustic models. The theoretical derivation clarifies the assumptions behind the segmentation‑free formulation, while the practical implementation addresses numerical issues and model peakiness. Empirical evidence across child and adult speech corpora shows that the new methods surpass existing GOP variants and recent end‑to‑end approaches, offering a robust, scalable solution for real‑time pronunciation feedback in CALL systems.