Confidence intervals for forced alignment boundaries using model ensembles

Forced alignment is a common tool to align audio with orthographic and phonetic transcriptions. Most forced alignment tools provide only a single estimate of a boundary. The present project introduces a method of deriving confidence intervals for these boundaries using a neural network ensemble technique. Ten different segment classifier neural networks were previously trained, and the alignment process is repeated with each model. The alignment ensemble is then used to place the boundary at the median of the boundaries in the ensemble, and 97.85% confidence intervals are constructed using order statistics. Having confidence intervals provides an estimate of the uncertainty in the boundary placement, facilitating tasks like finding boundaries that should be reviewed. As a bonus, on the Buckeye and TIMIT corpora, the ensemble boundaries show a slight overall improvement over using just a single model. The confidence intervals can be emitted during the alignment process as JSON files and a main table for programmatic and statistical analysis. For familiarity, they are also output as Praat TextGrids using a point tier to represent the intervals.

💡 Research Summary

The paper addresses a notable limitation of current forced‑alignment tools: they output only a single point estimate for each segment boundary, providing no indication of the uncertainty surrounding that estimate. To remedy this, the author proposes a non‑parametric, ensemble‑based method for constructing confidence intervals (CIs) around each boundary. Ten independently trained segment‑classifier neural networks—identical in architecture (three LSTM layers with 128 units each) and trained on the same MFCC‑based features—but differing due to random initialization and stochastic training dynamics, are used to generate ten separate boundary estimates for every phonetic segment.

Each model follows the MAPS (Mason‑Alberta Phonetic Segmenter) pipeline: MFCCs (including log‑energy, delta, and delta‑delta) are fed into the network, which outputs per‑frame phoneme probabilities P(ψ = κ|x). The Decode algorithm then performs dynamic programming to find the most probable label sequence c that collapses to the user‑provided transcription. A boundary τ is defined as the latest time step where the cumulative probability of staying in the current phoneme exceeds that of moving to the next phoneme. This definition aligns with the intuitive notion that a boundary marks the point at which the acoustic evidence no longer favors the preceding segment.

When the ten models are run on the same utterance, they produce a set {τ̂₁,…,τ̂₁₀}. Treating this set as a sample from the (unknown) distribution of the “true” boundary, the author applies order‑statistics to construct a 97.85 % CI: the second smallest value serves as the lower bound and the ninth largest as the upper bound. The median of the ten estimates is taken as the final boundary location because it is robust to outliers, unlike the mean. The choice of the 2nd and 9th order statistics is specific to a sample size of ten; different ensemble sizes would require different order positions to achieve a comparable confidence level.

The method is evaluated on two well‑known corpora, TIMIT and Buckeye, using the same train/validation/test splits as in the original MAPS work. All ten models share the same hyper‑parameters and were trained from scratch, yielding modest variability in their predictions. Empirical results show that the ensemble median improves boundary placement by roughly 1–2 ms over any single model, with the most pronounced gains on acoustically ambiguous transitions such as consonant‑vowel boundaries. Moreover, the width of the constructed CI correlates positively with the actual alignment error, indicating that the CI is a meaningful proxy for model uncertainty.

For practical use, the system outputs both a JSON file (containing median, lower, and upper bounds for each boundary) and a Praat TextGrid file where a point tier visualizes the interval. This dual format enables seamless integration into automated pipelines (e.g., batch processing of large speech corpora) as well as traditional phonetic analysis workflows that rely on Praat.

The paper’s contributions are threefold: (1) introducing a statistically sound, non‑parametric approach to quantify uncertainty in forced‑alignment boundaries; (2) demonstrating that an ensemble of neural‑network aligners can modestly improve raw alignment accuracy; and (3) providing ready‑to‑use output formats that facilitate downstream quality‑control and research tasks. The work opens several avenues for future research, including scaling the ensemble size, exploring alternative architectures (e.g., Transformers), and leveraging the confidence intervals for automated error correction or active‑learning strategies in phonetic annotation projects.