Easy Conditioning far beyond Gaussian

Multivariate Gaussian distributions enjoy Gaussian conditional distributions that makes conditioning easy: conditioning boils down to implementing analytical formulae for conditional means and covariances. For more general distributions, however, conditional distributions may not be available in analytical form and require demanding and approximate numerical approaches. Primarily motivatedby probabilistic imputation problems, we review and discuss families of multivariate distributions that do enjoy analytical conditioning, also providing a few counter-examples. Proving that trans-dimensional stability under conditioning extends to mixtures and transformations, we demonstrate that a broader class of multivariate distributions inherit easy conditioning properties. Building on this insight, we developed a generative method to estimate conditional distributions from data by first fitting a flexible joint distribution using copulas and then performing analytical conditioning in a latent space. In our applications, we specifically opt for Gaussian Mixture Copula Models (GMCM), comparing in turn various fitting strategies. Through simulations and real-world data experiments, we showcase the efficacy of our method in tasks involving conditional density estimation and data imputation. We also touch upon links to Gaussian process modelling and how stability by mixtures and transformations and mixtures carries over towards easy conditioning of non-Gaussian processes.

💡 Research Summary

The paper addresses a fundamental limitation of multivariate Gaussian models: while Gaussian distributions admit closed‑form conditional means and covariances, most realistic multivariate distributions do not, forcing practitioners to rely on costly numerical approximations for conditional inference and imputation. The authors introduce the concept of “trans‑dimensional stability” – a property of a family of probability distributions that remains closed under marginalisation and conditioning, regardless of the dimensionality of the variables involved.

First, they formalise this notion using precise operators for marginalisation (Marg) and conditioning (Cond) on a trans‑dimensional family (F=\bigcup_{d=1}^D F^{(d)}). A family is stable by marginalisation if (\text{Marg}(F)\subseteq F) and stable by conditioning if (\text{Cond}(F)\subseteq F). Classical examples such as the multivariate Gaussian, multivariate Student‑t, skew‑normal, skew‑Student‑t, and broader elliptically‑contoured families satisfy both properties. Counter‑examples are provided: a Student‑t family with a fixed degrees‑of‑freedom parameter (which changes under conditioning) and the multivariate q‑exponential distribution (whose conditional density no longer belongs to the same q‑family).

The authors then prove that these stability properties are preserved under finite mixtures and under invertible transformations. Theorem 2.1 shows that if a base family is stable, any finite mixture of its members is also stable, and similar arguments hold for transformations that map the latent space bijectively. This theoretical groundwork opens the door to constructing rich, non‑Gaussian models that still allow analytical conditioning.

Guided by this insight, the paper proposes a practical methodology for conditional density estimation based on copulas. Copulas separate marginal behaviour from dependence structure, and by selecting copula families that are themselves trans‑dimensionally stable, one can perform conditioning in a latent space where the joint distribution is Gaussian (or a mixture of Gaussians). The authors focus on the Gaussian Mixture Copula Model (GMCM). In a GMCM, each marginal is modelled (potentially non‑parametrically) and the dependence is captured by a finite mixture of multivariate Gaussian components. Parameter estimation is carried out via automatic differentiation, which improves over traditional EM‑based approaches in speed and accuracy.

The conditional inference procedure proceeds as follows: (1) transform observed conditioning variables (x_2) to uniform scores using their estimated marginal CDFs, (2) map these uniforms to a standard normal latent space via the probit function, (3) apply the analytical Gaussian (or Gaussian‑mixture) conditional formulas to obtain the latent distribution of the target variables (Z_1|Z_2), and (4) map back through the inverse probit and marginal inverse CDFs to obtain samples or density evaluations of (X_1|X_2). Because every step is analytic, the method avoids MCMC, variational approximations, or kernel density conditioning, resulting in substantial computational savings.

Empirical validation is performed on two datasets. The first is the classic wine dataset, where the authors model the joint distribution of alcohol content and malic acid concentration. Visualisations demonstrate that the GMCM captures the multimodal dependence and that conditional densities generated by the latent‑space conditioning match the empirical distribution. The second application involves a medical dataset with systematically missing values. Conditional density estimation is used for multivariate imputation, and performance is assessed with log‑score, CRPS, and KL‑divergence. Across all metrics, the GMCM‑based method outperforms kernel density estimators, Gaussian regression, and Bayesian network approaches.

The paper also discusses connections to Gaussian processes (GPs). Since a GP’s finite‑dimensional marginal distributions are multivariate Gaussian, they inherit trans‑dimensional stability. Consequently, non‑Gaussian process extensions (e.g., skew‑GP, t‑GP) that can be expressed as transformations or mixtures of Gaussian processes also retain analytical conditioning, suggesting a broader applicability of the theory.

In conclusion, the authors provide both a rigorous theoretical framework for identifying distribution families that permit easy conditioning and a concrete, scalable algorithm for conditional density estimation and imputation using copula‑based Gaussian mixture models. The work expands the toolbox for statisticians and machine‑learning practitioners dealing with complex, non‑Gaussian dependencies, and points toward future research on infinite mixtures, normalising flows, and non‑Gaussian process models for high‑dimensional conditional inference.