On micromodes in Bayesian posterior distributions and their implications for MCMC

We investigate the existence and severity of local modes in posterior distributions from Bayesian analyses. These are known to occur in posterior tails resulting from heavy-tailed error models such as those used in robust regression. To understand this phenomenon clearly, we consider in detail location models with Student-$t$ errors in dimension $d$ with sample size $n$. For sufficiently heavy-tailed data-generating distributions, extreme observations become increasingly isolated as $n \to \infty$. We show that each such observation induces a unique local posterior mode with probability tending to $1$. We refer to such a local mode as a micromode. These micromodes are typically small in height but their domains of attraction are large and grow polynomially with $n$. We then connect this posterior geometry to computation. We establish an Arrhenius law for the time taken by one-dimensional piecewise deterministic Monte Carlo algorithms to exit these micromodes. Our analysis identifies a phase transition where a misspecified and overly underdispersed model causes exit times to increase sharply, leading to a pronounced deterioration in sampling performance.

💡 Research Summary

The paper investigates a previously under‑appreciated source of multimodality in Bayesian posterior distributions that arises when heavy‑tailed error models are used. Focusing on a simple location model with Student‑t errors in dimension d and sample size n, the authors assume that the true data‑generating distribution P has polynomial tails of order β > 0. Under this assumption, extreme order statistics grow at rate n^{1/β} and become increasingly isolated from the bulk of the data as n increases.

The authors prove that each sufficiently extreme observation generates, with probability tending to one, a unique local posterior mode located within a √ν‑neighbourhood of the observation. They call such a mode a “micromode”. The existence of micromodes depends critically on the tail index β relative to the assumed Student‑t degrees of freedom ν. When β ≤ 1, the contribution of the extreme point dominates the empirical score in its neighbourhood, guaranteeing a micromode (Theorem 2.3). Conversely, if β > 1 the aggregated influence of the remaining n‑1 observations overwhelms the isolated point, preventing micromode formation (Theorem 2.2). The width of a micromode scales polynomially as W_n ≈ n^{1/β‑1}, while its height remains exponentially small in n.

To connect these geometric findings with computation, the paper analyses the one‑dimensional Zig‑Zag process (ZZP), a piecewise‑deterministic Monte Carlo (PDMC) algorithm. In a micromode the event rate, proportional to the absolute empirical score, is near zero, causing the process to linger. Using renewal theory, the authors derive an Arrhenius‑type law for the expected exit time τ_n:
 E