Sampling decomposable graphs using a Markov chain on junction trees

Full Bayesian computational inference for model determination in undirected graphical models is currently restricted to decomposable graphs, except for problems of very small scale. In this paper we develop new, more efficient methodology for such inference, by making two contributions to the computational geometry of decomposable graphs. The first of these provides sufficient conditions under which it is possible to completely connect two disconnected complete subsets of vertices, or perform the reverse procedure, yet maintain decomposability of the graph. The second is a new Markov chain Monte Carlo sampler for arbitrary positive distributions on decomposable graphs, taking a junction tree representing the graph as its state variable. The resulting methodology is illustrated with numerical experiments on three specific models.

💡 Research Summary

The paper addresses Bayesian model determination for undirected graphical models under the restrictive assumption that the conditional independence graph must be decomposable (i.e., chordal). While exact inference is feasible for small problems, existing methods become computationally prohibitive for moderate‑size graphs. The authors make two principal contributions that together enable more scalable inference.

First, they derive sufficient conditions under which one can either completely connect two disjoint complete vertex subsets or completely disconnect them, while guaranteeing that the resulting graph remains decomposable. Formally, let X and Y be two non‑empty, mutually disjoint subsets of vertices that are each complete in the current graph G and are completely disconnected (no edge between any x∈X and y∈Y). If X and Y belong to cliques that are adjacent in some junction tree representation of G, then adding all possible edges between X and Y yields a new graph G′ that is still decomposable (Proposition 1). The dual statement (Proposition 2) holds for the case where X∪Y already forms a complete subgraph and the two subsets lie in a single clique; removing all X–Y edges preserves decomposability. These results extend the classic single‑edge add/delete criteria (Giudici & Green, 1999) to multi‑edge moves, dramatically enlarging the admissible move set.

Second, the paper builds a Markov chain Monte Carlo (MCMC) sampler that treats a junction tree—a tree whose nodes are the cliques of a decomposable graph and whose edges correspond to separators—as the state variable, rather than the graph itself. For any decomposable graph G there are μ(G) distinct junction trees (Thomas & Green, 2009). The authors define a target distribution on junction trees, eπ(J)=π(G(J))/μ(G(J)), where π is the desired distribution on graphs (e.g., the posterior). The Markov chain is constructed to have eπ as its invariant distribution.

The sampling algorithm proceeds as follows:

1. Selection of candidate subsets – From the current junction tree J, pick two adjacent cliques C₁ and C₂. Randomly choose non‑empty subsets X⊆C₁ and Y⊆C₂ that satisfy the completeness requirements of the propositions. Because adjacency is checked directly on J, the selection automatically respects the sufficient conditions.

2. Multi‑edge proposal – Either add all edges between X and Y (complete‑connect) or delete all such edges (complete‑disconnect). This single proposal can change many edges simultaneously, unlike the single‑edge moves of earlier work.

3. Junction‑tree update – The new graph G′ obtained after the multi‑edge operation has a natural junction tree J′ that can be constructed by locally modifying J (splitting or merging the two adjacent cliques). No exhaustive search for a compatible tree is required. Optionally, a “randomisation” step can replace J′ by another junction tree representing the same G′, sampled uniformly from the μ(G′) possibilities, ensuring detailed balance across the whole equivalence class.

4. Metropolis–Hastings acceptance – Compute the usual acceptance ratio using the target π and the proposal probabilities (which are straightforward because the move set is symmetric). Accept or reject the (G′, J′) pair accordingly.

The authors compare this junction‑tree‑based sampler to the reversible‑jump sampler of Giudici & Green (1999). The key computational advantage is that the number of candidate vertex pairs considered at each step drops from O(|V|²) to O(|V|), because only vertices belonging to adjacent cliques in the current tree are eligible. Moreover, multi‑edge moves improve mixing: a single proposal can traverse a larger region of graph space, reducing autocorrelation.

Empirical evaluation is performed on three settings:

• Multivariate Gaussian models – Graphs of size 20–50 are sampled. The new sampler achieves 2–3× higher effective sample size (ESS) per unit CPU time compared with the Giudici‑Green method.

• Multinomial (categorical) models – Using Dirichlet priors on cell probabilities, the sampler demonstrates faster convergence, especially in higher dimensions where single‑edge moves become trapped.

• Large‑scale genetic linkage disequilibrium data – Over 100 000 markers are modeled with a decomposable graph. The junction‑tree sampler, integrated into a variant of the FitGMLD program, dramatically reduces memory consumption while preserving posterior inference quality.

The paper also discusses the limitation that the methodology relies on the decomposability assumption. Extending the approach to non‑chordal graphs would require additional approximations or alternative graph representations, which would re‑introduce computational penalties. Nonetheless, the authors argue that the sufficient‑condition framework and the junction‑tree state representation have broader relevance in computational graph theory and could inspire future work on approximate sampling for general graphs.

In summary, the work provides (1) a rigorous set of sufficient conditions for safe multi‑edge modifications of decomposable graphs, and (2) a novel, junction‑tree‑based MCMC algorithm that exploits those conditions to achieve substantial computational gains. This advances Bayesian structure learning for graphical models, making it feasible for substantially larger problems than previously possible.