Two-sample Testing with Block-wise Missingness in Multi-source Data

Multi-source and multi-modal datasets are increasingly common in scientific research, yet they often exhibit block-wise missingness, where entire modalities are systematically absent in some sources or no single source contains all modalities. This structured missingness poses major challenges for two-sample hypothesis testing. Standard approaches, such as imputation or complete-case analysis, may introduce bias or suffer efficiency loss, especially under missingness-not-at-random mechanisms. To address this challenge, we propose the Block-Pattern Enhanced Test, a general framework for constructing two-sample testing statistics that explicitly accounts for block-wise missingness. We show that the framework yields valid tests under a new condition allowing for missing-not-at-random mechanism. Building on this general framework, we further propose the Block-wise Rank In Similarity graph Edge-count (BRISE) test, which accommodate heterogeneous modalities using rank-based similarity graphs. Theoretically, we establish that the null distribution of BRISE converges to a $χ^2$ distribution, and that the test is consistent both in the standard asymptotic regime and in the high-dimensional low-sample-size setting under mild conditions. Simulation studies demonstrate that BRISE controls the type-I error rate and achieves strong power across a wide range of alternatives. Applications to two real-world datasets with block-wise missingness further illustrate the practical utility of the proposed method.

💡 Research Summary

The paper tackles a pressing problem in modern data analysis: two‑sample hypothesis testing when data are collected from multiple sources and modalities, yet entire modalities are missing for whole subsets of observations (block‑wise missingness). Traditional solutions such as imputation, complete‑case analysis, or naïve permutation tests either rely on strong assumptions (e.g., Missing At Random) or suffer severe power loss. To overcome these limitations, the authors introduce a general framework called the Block‑Pattern Enhanced Test (BPET). BPET first partitions the data into distinct missingness patterns, each defined by a binary vector indicating which modalities are observed. Within a pattern, only the shared modalities are used to compute pairwise dissimilarities; a flexible distance function ρₗ(·,·) and a normalization operator allow the method to accommodate Euclidean, network, functional, categorical, or any other data type. If two observations share no modality, the distance is set to zero (a simple pragmatic choice).

For each pattern α, a conventional two‑sample statistic (e.g., the RISE edge‑count statistic, Energy distance, MMD, etc.) is computed using only the observations that belong to that pattern. These pattern‑specific statistics are then aggregated—typically as a weighted sum that follows a χ² distribution under the null—yielding a global test statistic. The key theoretical insight is that, under the null hypothesis, it suffices that the two groups share the same conditional distribution within every pattern (F_X|Pα = F_Y|Pα). This condition is strictly weaker than the usual MCAR assumption and permits Missing‑Not‑At‑Random (MNAR) mechanisms where the pattern probabilities differ between groups.

Building on BPET, the authors propose a concrete test named BRISE (Block‑wise Rank‑In‑Similarity graph Edge‑count). BRISE constructs a rank‑weighted similarity graph for each pattern: edges connect observations that are close in the shared modalities, with weights derived from the ranks of the pairwise distances. The edge‑count differences between the two groups are then summed across patterns, producing a statistic that converges to a χ² distribution as the total sample size grows. The paper proves three main theoretical results: (1) the χ² null limit for BRISE, (2) consistency in both the classical asymptotic regime and the high‑dimensional low‑sample‑size (HDLSS) regime, and (3) the validity of a pattern‑wise permutation scheme. Standard permutation, which shuffles group labels across the whole dataset, fails when the marginal distribution of missingness patterns differs between groups (Theorem 1). The proposed pattern‑wise permutation respects the pattern‑specific counts (mα, nα) and restores exchangeability under the null (Theorem 2).

Simulation studies explore a wide range of settings: up to seven distinct missingness patterns, mixtures of Euclidean, network, and categorical modalities, and both balanced and highly unbalanced pattern frequencies. Across all scenarios, BRISE maintains the nominal type‑I error and exhibits substantially higher power than competing graph‑based tests (e.g., Friedman‑Rafsky, Energy, MMD) and classical parametric tests. The advantage is especially pronounced when missingness is MNAR or when the number of observed modalities varies dramatically across patterns.

Two real‑world applications illustrate practical relevance. In the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, the authors combine MRI, PET, genetics, and clinical variables, many of which are missing in a block‑wise fashion across sites. BRISE detects distributional differences between Alzheimer’s patients and cognitively normal controls that are missed by standard methods. A second study uses multi‑institution electronic health records where each hospital collects a different subset of labs and imaging studies; BRISE successfully identifies treatment‑group differences despite severe pattern imbalance. In both cases, the pattern‑wise permutation yields reliable p‑values, whereas naïve permutation would inflate false positives.

The paper also discusses limitations. When the number of modalities L is large, the number of possible patterns (up to 2ᴸ − 1) can become prohibitive, leading to very small pattern‑specific sample sizes and unstable variance estimates. The zero‑distance rule for completely non‑overlapping observations may discard useful indirect information; the authors suggest an extension using transitive similarity (see Supplementary Material). Finally, the choice of modality‑specific distance ρₗ influences power, so practitioners need to perform domain‑specific preprocessing or distance selection.

Overall, the work makes a substantial methodological contribution: BPET provides a principled, flexible framework for two‑sample testing under block‑wise missingness, and BRISE delivers a concrete, non‑parametric implementation that works for heterogeneous data, enjoys solid asymptotic guarantees, and performs well in finite‑sample and high‑dimensional regimes. The pattern‑wise permutation scheme is a novel solution to the exchangeability problem posed by structured missingness, and the empirical results convincingly demonstrate the utility of the approach for modern multi‑source biomedical studies.