Approximating Univariate Factored Distributions via Message-Passing Algorithms

Gaussian Mixture Models (GMMs) commonly arise in communication systems, particularly in bilinear joint estimation and detection problems. Although the product of GMMs is still a GMM, as the number of factors increases, the number of components in the resulting product GMM grows exponentially. To obtain a tractable approximation for a univariate factored probability density function (PDF), such as a product of GMMs, we investigate iterative message-passing algorithms. Based on Belief Propagation (BP), we propose a Variable Duplication and Gaussian Belief Propagation (VDBP)-based algorithm. The key idea of VDBP is to construct a multivariate measurement model whose marginal posterior is equal to the given univariate factored PDF. We then apply Gaussian BP (GaBP) to transform the global inference problem into local ones. Expectation propagation (EP) is another branch of message passing algorithms. In addition to converting the global approximation problem into local ones, it features a projection operation that ensures the intermediate functions (messages) belong to a desired family. Due to this projection, EP can be used to approximate the factored PDF directly. However, even if every factor is integrable, the division operation in EP may still cause the algorithm to fail when the mean and variance of a non-integrable belief are required. Therefore, this paper proposes two methods that combine EP with our previously proposed techniques for handling non-integrable beliefs to approximate univariate factored distributions.

💡 Research Summary

This paper addresses the challenge of efficiently estimating the statistics (mean and variance) of a univariate factored probability density function (PDF) that is expressed as a product of Gaussian mixture models (GMMs). In many communication‑system applications, especially in bilinear joint channel estimation and data detection, the likelihood can be written as a product of several simple GMM factors. While the product of GMMs is itself a GMM, the number of mixture components grows exponentially with the number of factors, making direct integration intractable.

To overcome this, the authors investigate two families of message‑passing algorithms: a Belief Propagation (BP)‑based method called Variable Duplication Gaussian Belief Propagation (VDBP) and two Expectation Propagation (EP)‑based methods that handle non‑integrable intermediate beliefs.

1. Variable Duplication Gaussian Belief Propagation (VDBP)
The key insight of VDBP is to transform the original univariate factored PDF into a multivariate linear measurement model. For each factor (f_n(\theta)) a duplicate variable (\theta_n) is introduced, and a linear constraint (\mathbf{A}\boldsymbol{\theta}=0) forces all duplicates to be equal. The matrix (\mathbf{A}\in\mathbb{R}^{(N-1)\times N}) satisfies (\mathbf{A}\mathbf{1}=0) (e.g., a trimmed Hadamard matrix). The resulting joint PDF becomes

\