Inference on the Significance of Modalities in Multimodal Generalized Linear Models

Despite the popular of multimodal statistical models, there lacks rigorous statistical inference tools for inferring the significance of a single modality within a multimodal model, especially in high-dimensional models. For high-dimensional multimodal generalized linear models, we propose a novel entropy-based metric, called the expected relative entropy, to quantify the information gain of one modality in addition to all other modalities in the model. We propose a deviance-based statistic to estimate the expected relative entropy, prove that it is consistent and its asymptotic distribution can be approximated by a non-central chi-squared distribution. That enables the calculation of confidence intervals and p-values to assess the significance of the expected relative entropy for a given modality. We numerically evaluate the empirical performance of our proposed inference tool by simulations and apply it to a multimodal neuroimaging dataset to demonstrate its good performance on various high-dimensional multimodal generalized linear models.

💡 Research Summary

The paper addresses a fundamental gap in multimodal statistical modeling: the lack of rigorous inference tools for assessing the contribution of an individual modality within a high‑dimensional multimodal generalized linear model (MGLM). The authors introduce a novel information‑theoretic metric called Expected Relative Entropy (ERE), denoted (H_m), which quantifies the average Kullback‑Leibler (KL) divergence between the full MGLM and an “oracle” reduced model that excludes the (m)‑th modality while retaining the true non‑zero coefficients from all other modalities. By definition, (H_m) is non‑negative and monotone with respect to adding modalities, making it a natural extension of classic goodness‑of‑fit measures such as (R^2) or pseudo‑(R^2).

To estimate (H_m) from data, the authors propose a deviance‑based estimator. Let (D(\beta)) be the standard GLM deviance; then (\hat H_m = D(\hat\beta_{-m}) - D(\hat\beta)), where (\hat\beta) and (\hat\beta_{-m}) are penalized maximum‑likelihood estimates (e.g., LASSO) for the full and reduced models, respectively. The paper proves that (\hat H_m) is consistent under standard high‑dimensional conditions (sparsity, restricted eigenvalue) and derives its asymptotic distribution: after scaling by (\sqrt{n}), (\hat H_m) converges to a non‑central chi‑squared distribution with degrees of freedom equal to the dimension of the excluded modality and non‑centrality parameter proportional to the true (H_m). Crucially, this result does not require variable‑selection consistency, allowing the method to be applied even when the LASSO fails to recover the exact support.

The authors illustrate the metric on several canonical GLMs. For linear regression with Gaussian covariates, a closed‑form expression links (H_m) to the conditional variance (\sigma^2_{m|{-}m}), showing that the new metric generalizes earlier work on modality contribution. For logistic, exponential, and Poisson regressions, the authors provide explicit formulas involving the model’s cumulant function; when analytic forms are unavailable, they suggest Monte‑Carlo approximation of the expectation over the covariate distribution.

Extensive simulations evaluate finite‑sample performance. Scenarios with (p) far exceeding (n) (e.g., (p=500, n=200)) and varying inter‑modality correlations demonstrate that the deviance‑based estimator accurately recovers (H_m) and yields well‑calibrated confidence intervals. Compared with variable‑level de‑biased LASSO tests, the proposed modality‑level test exhibits higher power, especially when the signal is spread across many weakly correlated variables within a modality.

A real‑world application uses a multimodal neuroimaging dataset (MRI, PET, CT) to predict Alzheimer’s disease status. The estimated (H_m) values rank PET as the most informative modality, followed by MRI and CT, with corresponding p‑values confirming statistical significance. These results illustrate how ERE can guide clinicians in selecting the most cost‑effective imaging techniques.

The discussion highlights several extensions: adapting ERE to survival models (Cox), mixed‑effects GLMs, and incorporating nonlinear interactions between modalities. Limitations include the computational burden of Monte‑Carlo integration for non‑Gaussian covariates and the reliance on correctly specified GLM link functions. Future work may explore faster approximation schemes and robustness to model misspecification.

In summary, the paper delivers a theoretically sound, practically implementable framework for modality‑level inference in high‑dimensional multimodal GLMs, filling a critical methodological void and opening avenues for more informed multimodal data integration across scientific domains.