Sharp convergence rates for Spectral methods via the feature space decomposition method

In this paper, we apply the Feature Space Decomposition (FSD) method developed in [LS24, GLS25, ALSS26] to obtain, under fairly general conditions, matching upper and lower bounds for the population excess risk of spectral methods in linear regression under the squared loss, for every covariance and every signal. This result enables us, for a given linear regression problem, to define a partial order on the set of spectral methods according to their convergence rates, thereby characterizing which spectral algorithm is superior for that specific problem. Furthermore, this allows us to generalize the saturation effect proposed in inverse problems and to provide necessary and sufficient conditions for its occurrence. Our method also shows that, under broad conditions, any spectral algorithm lacks a feature learning property, and therefore cannot overcome the barrier of the information exponent in problems such as single-index learning.

💡 Research Summary

The paper develops a unified, sharp analysis of spectral estimators for linear regression under squared loss, using the Feature Space Decomposition (FSD) framework introduced in earlier works