Combining chains of Bayesian models with Markov melding

A challenge for practitioners of Bayesian inference is specifying a model that incorporates multiple relevant, heterogeneous data sets. It may be easier to instead specify distinct submodels for each source of data, then join the submodels together. We consider chains of submodels, where submodels directly relate to their neighbours via common quantities which may be parameters or deterministic functions thereof. We propose chained Markov melding, an extension of Markov melding, a generic method to combine chains of submodels into a joint model. One challenge we address is appropriately capturing the prior dependence between common quantities within a submodel, whilst also reconciling differences in priors for the same common quantity between two adjacent submodels. Estimating the posterior of the resulting overall joint model is also challenging, so we describe a sampler that uses the chain structure to incorporate information contained in the submodels in multiple stages, possibly in parallel. We demonstrate our methodology using two examples. The first example considers an ecological integrated population model, where multiple data sets are required to accurately estimate population immigration and reproduction rates. We also consider a joint longitudinal and time-to-event model with uncertain, submodel-derived event times. Chained Markov melding is a conceptually appealing approach to integrating submodels in these settings.

💡 Research Summary

The paper addresses the practical problem of building a single Bayesian model that can accommodate several heterogeneous data sources. Rather than attempting to write a monolithic joint model from scratch, the authors propose to first develop separate sub‑models for each data set and then formally combine them. Existing “Markov melding” techniques allow such combination only when all sub‑models share a single common quantity. In many realistic applications the sub‑models are linked in a chain: model 1 shares a quantity with model 2, model 2 shares a (different) quantity with model 3, and so on. The authors therefore introduce Chained Markov melding, an extension of Markov melding that can handle an arbitrary number of sub‑models arranged in a linear chain.

The methodology proceeds in three conceptual steps. First, each sub‑model m (m = 1,…,M) has its own set of parameters θₘ, data Yₘ, and a subset φₘ of parameters that are common with its neighbours (φₘ = {φₘ₋₁∩ₘ, φₘ∩ₘ₊₁} for interior models). Because the priors on these common quantities differ across sub‑models, the authors replace each sub‑model’s marginal prior with a pooled prior p_pool(φ) that aggregates the original priors using a user‑chosen pooling function (linear or logarithmic pooling, including special cases such as Product‑of‑Experts or dictatorial pooling). This “marginal replacement” step is shown to minimise the Kullback–Leibler divergence from the original sub‑model while enforcing the common marginal.

Second, after marginal replacement each sub‑model now shares the same joint prior p_pool(φ). This enables the use of the classic Markov combination rule: the joint density of the whole system is the product of the pooled prior and the conditional densities of each sub‑model given the common quantities. The resulting joint model, called the chained melded model, automatically respects the dependence structure implied by the chain and correctly propagates uncertainty across all sub‑models.

Third, the authors develop a multi‑stage sampling algorithm that exploits the chain structure. In stage 1 each sub‑model is fitted independently (e.g., via MCMC, variational inference, or using existing software). In stage 2 the common quantities are updated sequentially using Gibbs‑type steps that condition on the most recent draws from neighbouring sub‑models. Because only adjacent models interact, many of the computations can be performed in parallel, dramatically reducing overall runtime. The algorithm also accommodates deterministic relationships (where a common quantity is a function of other parameters) and can reuse existing sub‑posterior samples or normal approximations when computational resources are limited.

Two applied examples illustrate the approach. The first is an integrated population model (IPM) for little owls, which combines capture‑recapture data, site‑occupancy counts, and nest‑record data. The three sub‑models share survival parameters (φ₁∩₂) and fecundity parameters (φ₂∩₃). Applying chained Markov melding reproduces the posterior estimates obtained from a fully specified joint IPM, while also allowing the authors to isolate the contribution of each data source and to reuse existing sub‑model code.

The second example concerns survival analysis with uncertain event times in intensive‑care patients. Here a hierarchical regression sub‑model provides a posterior distribution for the (unknown) time of respiratory failure, which is then used as a covariate in a survival sub‑model; a longitudinal sub‑model shares subject‑specific random effects with the survival model. Traditional joint models cannot incorporate the uncertainty of the event time and therefore risk biased inference. By treating the event‑time model as the first link in a chain, chained melding propagates its full posterior into the survival analysis, yielding less biased parameter estimates and appropriately wider credible intervals.

Overall, the paper contributes a flexible, theoretically grounded framework for Bayesian data fusion when sub‑models are naturally organised in a chain. It resolves the prior‑incompatibility issue via pooled marginal replacement, retains the full Bayesian uncertainty through Markov combination, and offers a practical, parallelisable inference algorithm. The methodology is broadly applicable to ecological modelling, health‑science joint modelling, and any domain where complex, multi‑source data must be integrated without sacrificing the rigor of a fully Bayesian analysis.