A note on target distribution ambiguity of likelihood-free samplers

Methods for Bayesian simulation in the presence of computationally intractable likelihood functions are of growing interest. Termed likelihood-free samplers, standard simulation algorithms such as Markov chain Monte Carlo have been adapted for this setting. In this article, by presenting generalisations of existing algorithms, we demonstrate that likelihood-free samplers can be ambiguous over the form of the target distribution. We also consider the theoretical justification of these samplers. Distinguishing between the forms of the target distribution may have implications for the future development of likelihood-free samplers.

💡 Research Summary

This paper investigates a subtle but important source of ambiguity in likelihood‑free (or approximate Bayesian computation) samplers: whether the algorithm is targeting the joint distribution of model parameters together with auxiliary simulated datasets, π_J(θ, t_{1:S} | t_y), or the marginal distribution of the parameters alone, π_M(θ | t_y). The authors first formalise the standard ABC set‑up: the observed data y are reduced to low‑dimensional summary statistics t_y = T(y); the intractable posterior π(θ | y) is replaced by π(θ | t_y), which remains intractable. To make inference feasible, one simulates auxiliary summary vectors t ∼ f(t | θ) from the model and weights them with a kernel K_h(t_y − t). By allowing S independent auxiliary draws t_{1:S}, they define a joint augmented posterior (Equation 3) and its marginal (Equation 2).

The central contribution is a systematic analysis of three families of likelihood‑free algorithms—rejection sampling, Metropolis–Hastings MCMC, and population‑based Sequential Monte Carlo (SMC)—under both target interpretations.

Rejection sampling (LF‑REJ): The algorithm draws (θ, t_{1:S}) from the prior predictive distribution and accepts with probability proportional to the average kernel weight (1/S)∑_s K_h(t_y − t_s). The authors prove that this acceptance rule is exactly the ratio π_J/π_prior, and after discarding the auxiliary draws it also yields draws from the marginal π_M. Hence the rejection sampler is unambiguously valid for both targets for any S ≥ 1.

Metropolis–Hastings MCMC (LF‑MCMC): When the marginal target is assumed, the acceptance probability involves the ratio of two Monte‑Carlo estimates of π_M, i.e. ˆπ_M(θ′)/ˆπ_M(θ). Because E