Minimally Discrete and Minimally Randomized p-Values

In meta analysis, multiple hypothesis testing and many other methods, p-values are utilized as inputs and assumed to be uniformly distributed over the unit interval under the null hypotheses. If data used to generate p-values have discrete distributions then either natural, mid- or randomized p-values are typically utilized. Natural and mid-p-values can allow for valid, albeit conservative, downstream methods since under the null hypothesis they are dominated by uniform distributions in the stochastic and convex order, respectively. Randomized p-values need not lead to conservative procedures since they permit a uniform distributions under the null hypotheses through the generation of independent auxiliary variates. However, the auxiliary variates necessarily add variation to procedures. This manuscript introduces and studies minimally discrete'' (MD) natural p-values, MD mid-p-values and minimally randomized’’ (MR) p-values. It is shown that MD p-values dominate their non-MD counterparts in the stochastic and convex order, and hence lead to less conservative, yet still valid, downstream methods. Likewise, MR p-values dominate their non-MR counterparts in that they are still uniformly distributed under the null hypotheses, but the added variation attributable to the independently generated auxiliary variate is smaller. It is anticipated that results here will facilitate the construction of new meta-analysis and multiple testing methods via more efficient p-value construction, and facilitate theoretical study of existing and new methods by establishing gold standards for addressing the unavoidable detrimental ``discreteness effect’'.

💡 Research Summary

The manuscript tackles a fundamental problem in statistical inference with discrete data: the “discreteness effect” that makes ordinary p‑values deviate from the ideal uniform distribution under the null hypothesis. When the test statistic takes only a finite set of values, the usual construction of natural p‑values (based on Pr₀{T≥t}) yields a distribution that stochastically dominates Uniform(0,1), while mid‑p‑values (which average the probability of equality) are sub‑uniform in the convex order. Randomized p‑values restore exact uniformity by introducing an auxiliary uniform variable U, but the extra randomness inflates variability and harms reproducibility.

The authors propose two new families of p‑values that are optimal with respect to two complementary criteria. “Minimally discrete” (MD) p‑values are defined for both natural and mid‑p‑value settings. An MD natural p‑value is the smallest α such that the associated non‑randomized test function attains size α; equivalently, it is the largest p‑value that still respects the unbiasedness constraint. This construction yields a p‑value that dominates the ordinary natural p‑value in the stochastic order, i.e., it is less conservative while still guaranteeing validity. Similarly, an MD mid‑p‑value is obtained by minimizing α under the convex‑order constraint, producing a p‑value that dominates the traditional mid‑p‑value in the convex order. The paper proves these dominance results using Lemma 1 (which shows that any p‑value linear in the auxiliary variable is sub‑uniform) and Corollary 1, establishing that the MD versions are the sharpest possible under the respective ordering.

For the randomized case, the authors introduce “minimally randomized” (MR) p‑values. An MR p‑value retains the exact Uniform(0,1) distribution under H₀ but reduces the variance contributed by the auxiliary uniform variable. This is achieved by restricting the randomization to the smallest interval consistent with the test’s rejection region: for each observed statistic t, the randomization is confined to the interval (