Additive Models Explained: A Computational Complexity Approach

Generalized Additive Models (GAMs) are commonly considered interpretable within the ML community, as their structure makes the relationship between inputs and outputs relatively understandable. Therefore, it may seem natural to hypothesize that obtaining meaningful explanations for GAMs could be performed efficiently and would not be computationally infeasible. In this work, we challenge this hypothesis by analyzing the computational complexity of generating different explanations for various forms of GAMs across multiple contexts. Our analysis reveals a surprisingly diverse landscape of both positive and negative complexity outcomes. Particularly, under standard complexity assumptions such as P!=NP, we establish several key findings: (1) in stark contrast to many other common ML models, the complexity of generating explanations for GAMs is heavily influenced by the structure of the input space; (2) the complexity of explaining GAMs varies significantly with the types of component models used - but interestingly, these differences only emerge under specific input domain settings; (3) significant complexity distinctions appear for obtaining explanations in regression tasks versus classification tasks in GAMs; and (4) expressing complex models like neural networks additively (e.g., as neural additive models) can make them easier to explain, though interestingly, this benefit appears only for certain explanation methods and input domains. Collectively, these results shed light on the feasibility of computing diverse explanations for GAMs, offering a rigorous theoretical picture of the conditions under which such computations are possible or provably hard.

💡 Research Summary

The paper investigates the computational complexity of generating a variety of explanations for Generalized Additive Models (GAMs). While GAMs are widely regarded as interpretable because their prediction is a sum of univariate component functions, the authors question whether this interpretability translates into efficient algorithms for producing concrete explanations. They conduct a systematic three‑dimensional analysis that crosses (i) the type of input domain (enumerable discrete, general discrete, continuous), (ii) the class of component models used inside the GAM (classical spline‑based smooth GAMs, Neural Additive Models (NAMs) with ReLU networks, and Explainable Boosting Machines (EBMs) that employ boosted tree ensembles), and (iii) the explanation task (minimum sufficient reasons, minimum contrastive reasons, Shapley value attributions, and global feature redundancy).

For each of the 3 × 3 × 4 = 36 configurations they formulate the explanation problem as a decision/query problem and classify its computational difficulty into standard complexity classes: polynomial‑time (PTIME), pseudo‑polynomial time (when model coefficients are given in unary), NP‑Hard, #P‑Hard, Σ₂ᴾ‑Hard, or Δ₂ᴾ/Σ₂ᴾ. The results are summarized in a comprehensive table (Table 1 in the paper).

Key findings include:

1. Input‑domain sensitivity – Most explanation tasks become dramatically harder when moving from an enumerable discrete domain (where each feature has a constant‑size value set) to a general discrete or continuous domain. For example, computing minimum sufficient reasons (MSR) and minimum contrastive reasons (MCR) is PTIME in the enumerable case but NP‑Hard in continuous settings. Conversely, detecting global feature redundancy is easier in the continuous case (PTIME) than in discrete cases. This domain‑dependent behavior is unique to additive models; non‑additive models such as decision trees or neural networks do not exhibit such stark variations.

2. Component‑model impact – Smooth GAMs built from low‑degree splines admit efficient algorithms for almost all explanation types because the component functions are simple polynomials. NAMs and EBMs, which embed non‑linear neural nets or boosted trees, cause many explanation problems to jump to NP‑Hard, Σ₂ᴾ‑Hard, or #P‑Hard in the general discrete and continuous domains. The complexity gap between spline‑based and neural/boosted components disappears in the enumerable discrete setting, where all three model families behave similarly.

3. Regression vs. classification split – The paper uncovers a striking separation for Shapley value explanations: in regression tasks SHAP can be computed in PTIME (thanks to linearity of expectations), whereas in binary classification the step function makes SHAP #P‑Hard. No such split appears for the other explanation types.

4. Additive reformulation benefits – Non‑additive models that are known to be hard to explain (standard neural networks, tree ensembles) become easier when expressed in an additive form (NAMs, EBMs). For certain domains and explanation tasks the additive version falls into PTIME or pseudo‑PTIME, whereas the original formulation remains at least NP‑Hard. However, this benefit is not universal; for instance, SHAP on continuous classification remains #P‑Hard even for additive models.

5. Pseudo‑polynomial tractability – When the model coefficients β_j are encoded in unary (i.e., with limited precision), several problems that are #P‑Hard in the binary encoding become solvable in pseudo‑polynomial time. This suggests that reducing coefficient precision—common in model quantization—can dramatically improve the feasibility of explanation computation.

The authors discuss practical implications: designers of GAM‑based systems should first decide which explanation type they need and what input domain their data inhabits. If the target explanation is a minimum sufficient reason on continuous data, a spline‑based GAM is preferable; if the data are categorical with a small fixed alphabet, even NAMs or EBMs can be used without incurring prohibitive computational costs. For SHAP, regression tasks are tractable while classification tasks may require approximations (e.g., KernelSHAP) or alternative attribution methods.

Finally, the paper outlines future directions, such as extending the analysis to multi‑dimensional outputs, quantifying trade‑offs between explanation accuracy and computational hardness, and exploring algorithmic techniques that exploit problem structure (e.g., dynamic programming on unary‑encoded coefficients). Overall, the work provides a rigorous “complexity map” for GAM explanations, clarifying when the promise of interpretability aligns with algorithmic feasibility and when it does not.