Reliable Explanations or Random Noise? A Reliability Metric for XAI

In recent years, explaining decisions made by complex machine learning models has become essential in high-stakes domains such as energy systems, healthcare, finance, and autonomous systems. However, the reliability of these explanations, namely, whether they remain stable and consistent under realistic, non-adversarial changes, remains largely unmeasured. Widely used methods such as SHAP and Integrated Gradients (IG) are well-motivated by axiomatic notions of attribution, yet their explanations can vary substantially even under system-level conditions, including small input perturbations, correlated representations, and minor model updates. Such variability undermines explanation reliability, as reliable explanations should remain consistent across equivalent input representations and small, performance-preserving model changes. We introduce the Explanation Reliability Index (ERI), a family of metrics that quantifies explanation stability under four reliability axioms: robustness to small input perturbations, consistency under feature redundancy, smoothness across model evolution, and resilience to mild distributional shifts. For each axiom, we derive formal guarantees, including Lipschitz-type bounds and temporal stability results. We further propose ERI-T, a dedicated measure of temporal reliability for sequential models, and introduce ERI-Bench, a benchmark designed to systematically stress-test explanation reliability across synthetic and real-world datasets. Experimental results reveal widespread reliability failures in popular explanation methods, showing that explanations can be unstable under realistic deployment conditions. By exposing and quantifying these instabilities, ERI enables principled assessment of explanation reliability and supports more trustworthy explainable AI (XAI) systems.

💡 Research Summary

The paper addresses a critical gap in the field of explainable artificial intelligence (XAI): while many attribution methods such as SHAP, Integrated Gradients (IG), DeepLIFT, LIME, and others are widely used, there is no systematic way to assess whether the explanations they produce remain stable under realistic, non‑adversarial changes that occur in real‑world deployments. The authors argue that “reliability” – the property that explanations should not fluctuate dramatically when inputs are slightly perturbed, when redundant features are present, when a model is modestly updated, or when the data distribution shifts mildly – is a prerequisite for trustworthy XAI, especially in high‑stakes domains like healthcare, finance, energy, and autonomous systems.

To formalize this notion, the authors introduce four reliability axioms:

• Axiom 1 (Stability under Small Perturbations) – Explanations must be locally Lipschitz with respect to the input, guaranteeing that a bounded input noise ε leads to a bounded change in the attribution vector.

• Axiom 2 (Redundancy‑Collapse Consistency) – When two features become perfectly correlated (e.g., temperature in Celsius vs. Fahrenheit), collapsing the redundant pair into a single effective feature should cause the explanation to converge to the same value; mathematically, the distance between the collapsed explanation and the original explanation must vanish as the redundancy parameter α → 1.

• Axiom 3 (Smooth Evolution across Model Updates) – Small, non‑adversarial transformations of the model (e.g., fine‑tuning, pruning) should induce only a small drift Δ in the explanation. A monotone decreasing function ψ maps this drift to a reliability score ERI = ψ(Δ) that approaches 1 as Δ → 0.

• Axiom 4 (Distributional Robustness) – If two data distributions P and P′ are close in Wasserstein‑1 distance (≤ ε), the expected explanations under each distribution should also be close, bounded by a Lipschitz constant L_E times ε.

Based on these axioms, the authors propose the Explanation Reliability Index (ERI), a family of metrics that evaluates an explainer on each axiom separately (ERI‑S, ERI‑R, ERI‑M, ERI‑D) and optionally aggregates them into a single scalar score. For sequential models (LSTMs, GRUs, Transformers, temporal CNNs) they introduce ERI‑T, a temporal reliability measure that quantifies how smoothly explanations evolve over time, independent of prediction smoothness.

The theoretical contributions include Lipschitz‑type bounds for A1, a formal collapse operator and convergence proof for A2, a continuity‑based reliability function for A3, and a Wasserstein‑based bound for A4. The paper also discusses practical estimation of the required constants (C_E, L_E) and distance functions (L2, cosine, rank‑based) and provides guidance on computational considerations.

To validate the framework, the authors build ERI‑Bench, a benchmark that systematically stress‑tests explanation reliability across four axes: synthetic perturbations (noise, controlled redundancy), real‑world time‑series datasets (EEG microstates, UCI HAR, Norwegian load forecasting), and vision data (CIFAR‑10). They evaluate a broad set of explainers: SHAP, Integrated Gradients, Gradient×Input, DeepLIFT, LIME, SmoothGrad, as well as dependence‑based methods such as Mutual Information (MI), Hilbert–Schmidt Independence Criterion (HSIC), and a recently proposed MCIR method.

Experimental findings reveal pervasive reliability failures in the traditional attribution methods. SHAP and IG, while often faithful in the sense of matching model output, score low on ERI‑S (≈ 0.4–0.5) under small input noise and on ERI‑R (≈ 0.3) when redundant features are introduced. Gradient‑based methods exhibit moderate stability to noise but degrade sharply under feature redundancy and temporal shifts (ERI‑T ≈ 0.35). LIME is highly sensitive to sampling noise, leading to poor scores across all axioms. In contrast, dependence‑based methods (MI, HSIC, MCIR) consistently achieve higher reliability (ERI scores ranging from 0.7 to 0.85) across all four axes, suggesting that measuring statistical dependence rather than gradient or game‑theoretic contributions yields more robust explanations under realistic variations.

Beyond benchmarking, the authors demonstrate practical uses of ERI: selecting model checkpoints that maximize reliability without sacrificing predictive accuracy, guiding feature engineering to reduce redundancy, and informing the design of explanation‑aware training pipelines (e.g., adding a regularizer that penalizes large ERI‑S violations). They acknowledge limitations, notably the computational overhead of estimating Lipschitz constants in high‑dimensional spaces and the fact that ERI addresses reliability but not other important XAI dimensions such as faithfulness, causality, or human interpretability.

In conclusion, the paper makes three major contributions: (1) a formal, axiomatic definition of explanation reliability; (2) the ERI family of metrics, including the novel temporal component ERI‑T; and (3) the ERI‑Bench benchmark that reveals systematic instability in widely used XAI methods. By providing a quantitative, theory‑backed reliability signal, the work equips researchers and practitioners with a new tool to evaluate, compare, and improve explainers, moving XAI closer to the trustworthiness required for deployment in safety‑critical applications.