The Algebraic Complexity of Maximum Likelihood Estimation for Bivariate Missing Data

We study the problem of maximum likelihood estimation for general patterns of bivariate missing data for normal and multinomial random variables, under the assumption that the data is missing at random (MAR). For normal data, the score equations have nine complex solutions, at least one of which is real and statistically significant. Our computations suggest that the number of real solutions is related to whether or not the MAR assumption is satisfied. In the multinomial case, all solutions to the score equations are real and the number of real solutions grows exponentially in the number of states of the underlying random variables, though there is always precisely one statistically significant local maxima.

💡 Research Summary

The paper investigates the algebraic complexity of maximum‑likelihood estimation (MLE) when data are missing at random (MAR) in a bivariate setting. Two families of distributions are considered: continuous bivariate normal and discrete bivariate multinomial. The authors adopt an algebraic‑geometric viewpoint, focusing on the number of complex solutions to the likelihood equations (the so‑called ML‑degree) and on how the real solutions relate to the validity of the MAR assumption.

Problem formulation.
Let (X₁,X₂) be a random vector. For n fully observed pairs, r observations of X₁ only, and s observations of X₂ only, the observed‑data log‑likelihood is the sum of the log‑densities for the three groups. Under MAR the missingness mechanism does not affect the likelihood, so the usual MLE problem applies.

Bivariate normal case.
The authors re‑parameterize the covariance matrix Σ by its inverse Γ = (γ₁₁ γ₁₂; γ₁₂ γ₂₂). After eliminating constants, the log‑likelihood becomes a rational function of μ = (μ₁,μ₂) and Γ. Differentiating yields five polynomial score equations (two for the means, three for the entries of Γ). Using Gröbner‑basis computations in Singular, they show that for generic data the system has exactly nine complex solutions; this number is the ML‑degree of the model. Because complex solutions of real polynomial systems appear in conjugate pairs, at least one solution is real. Moreover, the log‑likelihood tends to –∞ when any parameter diverges or when Σ approaches the boundary of the positive‑definite cone, guaranteeing at least one real critical point that is a local maximum in the admissible parameter space ℝ² × PD₂.

The authors then explore how many of the nine solutions are real. Empirical experiments reveal a striking pattern: when data are truly MAR (or MCAR) the likelihood equations typically have a single real solution, which is the unique global maximum. When the missingness is not at random (NMAR) or when the data are generated from a mismatched model, three, five, seven, or even nine real solutions can appear. In many NMAR simulations three real solutions arise, two of which are local maxima. This suggests that the number of real solutions can serve as a diagnostic for how well the MAR assumption fits the data. Consequently, the EM algorithm, which converges to a local maximum, may be misled if multiple maxima exist; careful initialization or algebraic checks become advisable.

Bivariate multinomial case.
Here X₁ takes values in {1,…,m} and X₂ in {1,…,n}. The observed data consist of a full contingency table T together with marginal count vectors R (for X₁ only) and S (for X₂ only). The log‑likelihood is linear in the cell probabilities p_{ij} subject to non‑negativity and Σ p_{ij}=1. The score equations are linear, so every solution is real. The number of solutions equals the number of bounded regions in the hyperplane arrangement defined by the equations p_{ij}=0, p_{i+}=0, p_{+j}=0 intersected with the simplex Σ p_{ij}=1. By invoking known results on hyperplane arrangements, the authors derive a closed combinatorial formula for the ML‑degree:

\