Empirical Likelihood-Based Fairness Auditing: Distribution-Free Certification and Flagging

Machine learning models in high-stakes applications, such as recidivism prediction and automated personnel selection, often exhibit systematic performance disparities across sensitive subpopulations, raising critical concerns regarding algorithmic bias. Fairness auditing addresses these risks through two primary functions: certification, which verifies adherence to fairness constraints; and flagging, which isolates specific demographic groups experiencing disparate treatment. However, existing auditing techniques are frequently limited by restrictive distributional assumptions or prohibitive computational overhead. We propose a novel empirical likelihood-based (EL) framework that constructs robust statistical measures for model performance disparities. Unlike traditional methods, our approach is non-parametric; the proposed disparity statistics follow asymptotically chi-square or mixed chi-square distributions, ensuring valid inference without assuming underlying data distributions. This framework uses a constrained optimization profile that admits stable numerical solutions, facilitating both large-scale certification and efficient subpopulation discovery. Empirically, the EL methods outperform bootstrap-based approaches, yielding coverage rates closer to nominal levels while reducing computational latency by several orders of magnitude. We demonstrate the practical utility of this framework on the COMPAS dataset, where it successfully flags intersectional biases, specifically identifying a significantly higher positive prediction rate for African-American males under 25 and a systemic under-prediction for Caucasian females relative to the population mean.

💡 Research Summary

The paper introduces a novel empirical‑likelihood‑based fairness auditing framework (ELFA) that simultaneously addresses two core tasks in algorithmic fairness: certification (verifying that a model satisfies predefined fairness constraints) and flagging (identifying subpopulations that exceed a tolerated disparity). Traditional auditing methods often rely on strong distributional assumptions (e.g., identical distributions across groups) or computationally intensive resampling techniques such as bootstrap or permutation tests, which limit scalability and robustness. ELFA circumvents these limitations by leveraging the empirical likelihood (EL) methodology, a non‑parametric approach that constructs data‑adaptive likelihood ratios without explicit variance estimation or pivot quantities.

The authors formalize group‑wise performance disparity as ε_G = E