Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis

Machine learning models have achieved widespread success but often inherit and amplify historical biases, resulting in unfair outcomes. Traditional fairness methods typically impose constraints at the prediction level, without addressing underlying biases in data representations. In this work, we propose a principled framework that adjusts data representations to balance predictive utility and fairness. Using sufficient dimension reduction, we decompose the feature space into target-relevant, sensitive, and shared components, and control the fairness-utility trade-off by selectively removing sensitive information. We provide a theoretical analysis of how prediction error and fairness gaps evolve as shared subspaces are added, and employ influence functions to quantify their effects on the asymptotic behavior of parameter estimates. Experiments on both synthetic and real-world datasets validate our theoretical insights and show that the proposed method effectively improves fairness while preserving predictive performance.

💡 Research Summary

This paper tackles the pervasive problem of bias amplification in machine learning by shifting the focus from post‑hoc prediction constraints to the very representation of the data. Building on sufficient dimension reduction (SDR) theory, the authors decompose the high‑dimensional feature space X∈ℝ^p into three orthogonal subspaces: (i) directions that are informative for the target Y but orthogonal to the sensitive attribute Z, (ii) directions that are shared between Y and Z, and (iii) directions irrelevant to both. The central subspaces for Y and Z, denoted S_Y|X and S_Z|X, are assumed to intersect in a subspace of dimension s (possibly zero). By explicitly identifying this intersection, the method can gradually re‑introduce the shared directions, thereby providing a controllable trade‑off between predictive utility and fairness.

Formally, the authors define a family of projection matrices P(m)=e_B e_Bᵀ+Φ_m Φ_mᵀ for m∈{0,…,s}, where e_B spans the part of S_Y|X orthogonal to S_Z|X (the “fair” component) and Φ_m contains the first m shared basis vectors. The transformed representation Ξ(m)=P(m)X is then used to construct the Bayes optimal predictor ˜f(m)=E