A prediction interval for the population-wise error rate

We construct an asymptotic prediction interval for the population-wise error rate (PWER), which is a multiple type I error criterion for clinical trials with overlapping patient populations. The PWER is the probability that a randomly selected patient will receive an ineffective treatment. It must usually be estimated due to unknown population strata sizes, such that only an estimate can be controlled at the given significance level. We apply the delta method to find a prediction interval for the resulting true PWER, we demonstrate by simulations that the interval has the required coverage probability, and illustrate the approach with real data examples.

💡 Research Summary

The paper addresses a practical gap in the application of the population‑wise error rate (PWER), a multiple‑testing error metric designed for clinical trials that involve overlapping patient subpopulations. While the PWER can be controlled at a pre‑specified significance level α by choosing appropriate critical values for each hypothesis, the required stratum prevalence vector π (the relative sizes of the disjoint strata P_J) is usually unknown. In practice one estimates π by the multinomial maximum‑likelihood estimator (\hat\pi_J = n_J/N), where n_J is the observed number of patients in stratum J and N the total sample size. The authors previously showed that plugging (\hat\pi) into the critical‑value function yields an estimated PWER that converges almost surely to α as N → ∞, and that in finite samples the true PWER is typically very close to α. However, a single study may still exhibit random deviations, and quantifying this uncertainty is essential for robust trial design.

The core contribution of the article is the derivation of an asymptotic prediction interval for the true PWER in a given study. The authors exploit the well‑known asymptotic normality of the multinomial MLE: \