Another Marcenko-Pastur law for Kendall's tau

Bandeira et al. (2017) show that the eigenvalues of the Kendall correlation matrix of $n$ i.i.d. random vectors in $\mathbb{R}^p$ are asymptotically distributed like $1/3 + (2/3)Y_q$, where $Y_q$ has a Marčenko-Pastur law with parameter $q=\lim(p/n)$ if $p, n\to\infty$ proportionately to one another. Here we show that another Marčenko-Pastur law emerges in the “ultra-high dimensional” scaling limit where $p\sim q’, n^2/2$ for some $q’>0$: in this quadratic scaling regime, Kendall correlation eigenvalues converge weakly almost surely to $(1/3)Y_{q’}$.

💡 Research Summary

This paper investigates the asymptotic spectral behavior of the Kendall correlation matrix τ when the dimensionality p grows quadratically with the sample size n, i.e., p ≈ (q′/2) n² for a fixed positive constant q′. In the classical “linear” regime where p and n are of the same order, Bandeira et al. (2017) showed that the empirical spectral distribution (ESD) of τ converges to an affine transformation of the Marčenko–Pastur (MP) law: (1/3)+(2/3) Y_q, with Y_q ∼ MP(q) and q = lim p/n. The present work demonstrates that, under the ultra‑high‑dimensional scaling, a different MP law emerges: the ESD of τ converges almost surely to (1/3) Y_{q′}, where Y_{q′} has the standard MP distribution with parameter q′ = lim 2p/(n(n−1)).

The authors start by recalling the Hoeffding decomposition of the sign function that underlies Kendall’s tau. For each pair of observations (i,j) they write the centered sign variable as a sum of two conditional expectations and a residual term that is mean‑zero and uncorrelated with the others. This yields a matrix decomposition
τ_n = H_n + A_n,
where H_n = ∑{i<j} \barΘ^{(ij)}\barΘ^{(ij)⊤} collects the centered residuals and A_n = ∑{i<j} A^{(ij)} contains the deterministic “intercept” part. In the linear regime, A_n produces the MP component (2/3) Y_q while H_n contributes the constant (1/3)I_p.

In the quadratic regime, the ratio q = p/n tends to infinity, so the contribution of A_n vanishes (it becomes asymptotically negligible), and the spectral bulk is governed entirely by H_n. The first technical step (Proposition 1) shows that the ESDs of τ_n and H_n are asymptotically identical, i.e., ‖F_{τ_n}−F_{H_n}‖_∞ → 0 a.s. Consequently, the problem reduces to studying the limiting spectral distribution (LSD) of H_n.

The authors adopt the Stieltjes‑transform (or “cavity”) method, mirroring the classical proof for Wishart matrices. They define the empirical Stieltjes transform g_p(z) = (p⁻¹) tr (zI − H_n)⁻¹ and prove two key statements: (1) g_p(z) concentrates around its expectation, i.e., g_p(z) − E