Concentration Inequalities for Suprema of Empirical Processes with Dependent Data via Generic Chaining with Applications to Statistical Learning

This paper develops a general concentration inequality for the suprema of empirical processes with dependent data. The concentration inequality is obtained by combining generic chaining with a coupling-based strategy. Our framework accommodates high-dimensional and heavy-tailed (sub-Weibull) data. We demonstrate the usefulness of our result by deriving non-asymptotic predictive performance guarantees for empirical risk minimization in regression problems with dependent data. In particular, we establish an oracle inequality for a broad class of nonlinear regression models and, as a special case, a single-layer neural network model. Our results show that empirical risk minimzaton with dependent data attains a prediction accuracy comparable to that in the i.i.d. setting for a wide range of nonlinear regression models.

💡 Research Summary

This paper develops a general concentration inequality for the supremum of empirical processes when the underlying observations are dependent. The authors focus on sequences that satisfy β‑mixing (absolute regularity) conditions and allow for high‑dimensional, heavy‑tailed data modeled as sub‑Weibull(α) random variables (including sub‑Gaussian (α=2) and sub‑Exponential (α=1) as special cases).

The methodological core combines two powerful tools: (i) generic chaining, originally introduced by Talagrand, which yields sharp bounds on suprema via the γ‑functional γα(Θ) that captures the metric complexity of the index set Θ; and (ii) a coupling technique based on Merlevède and Peligrad’s results, which approximates the dependent sequence {Zt} by an i.i.d. copy {Z*t} while controlling the approximation error through the β‑mixing coefficients.

Two high‑level assumptions are imposed. The increment condition (A.1) requires that the sub‑Weibull norm of the centered difference gθ1(Zt)−gθ2(Zt) is bounded by a constant times the metric distance dΘ(θ1,θ2). This Lipschitz‑type condition enables the chaining construction. The coupling condition (A.2) demands that the expected supremum of |gθ(Zt)−gθ(Z*t)| can be bounded by the Lr‑norm of a metric dZ between Zt and its copy, and that (Z,dZ) is a Polish space. These conditions together allow the authors to replace the dependent data by an i.i.d. surrogate and to quantify the resulting bias via the β‑mixing coefficients.

The main result (Theorem 2.1) states that for any effective sample size n≤T, any ε1≥2 and any ε2>0, the event
\