A Framework for the Bayesian Calibration of Complex and Data-Scarce Models in Applied Sciences

In this work, we review the theory involved in the Bayesian calibration of complex computer models, with particular emphasis on their use for applications involving computationally expensive simulations and scarce experimental data. In the article, we present a unified framework that incorporates various Bayesian calibration methods, including well-established approaches. Furthermore, we describe their implementation and use with a new, open-source Python library, ACBICI (A Configurable BayesIan Calibration and Inference Package). All algorithms are implemented with an object-oriented structure designed to be both easy to use and readily extensible. In particular, single-output and multiple-output calibration are addressed in a consistent manner. The article completes the theory and its implementation with practical recommendations for calibrating the problems of interest. These guidelines – currently unavailable in a unified form elsewhere – together with the open-source Python library, are intended to support the reliable calibration of computational codes and models commonly used in engineering and related fields. Overall, this work aims to serve both as a comprehensive review of the statistical foundations and (computational) tools required to perform such calculations, and as a practical guide to Bayesian calibration with modern software tools.

💡 Research Summary

This paper presents a comprehensive review of Bayesian calibration theory, focusing on the Kennedy‑O’Hagan (KOH) framework, and introduces a new open‑source Python library called ACBICI (A Configurable Bayesian Calibration and Inference Package). The authors first revisit the mathematical foundations of KOH, which treat model parameters and a model‑experiment discrepancy function as random variables, allowing full posterior inference that captures uncertainty in both parameters and model inadequacy. Experimental measurement error is modeled as zero‑mean Gaussian noise, with the variance optionally treated as an additional unknown to be inferred.

Four calibration scenarios are defined:

• Type A – simple calibration of inexpensive models without surrogate or discrepancy components; only priors on parameters are required.
• Type B – calibration of expensive simulators using a fast surrogate (Gaussian‑process emulator) to replace the original code during inference.
• Type C – inclusion of a discrepancy function when the underlying model is structurally insufficient.
• Type D – the most general case that combines surrogate modeling, discrepancy, and experimental error, and supports simultaneous multi‑output calibration.

The paper emphasizes that multi‑output calibration must retain the joint probabilistic structure, modeling correlations among outputs via a multivariate Gaussian‑process. This avoids the statistical inconsistency of treating each output independently.

ACBICI implements the entire KOH pipeline with an object‑oriented design. Core classes (Model, Surrogate, Discrepancy, Prior, Likelihood, Sampler) can be mixed and matched, enabling users to construct custom calibration workflows with minimal code. The library interfaces with modern probabilistic programming back‑ends such as PyMC3, NumPyro, and TensorFlow Probability, providing Hamiltonian Monte Carlo (HMC), the No‑U‑Turn Sampler (NUTS), and also a variational inference (VI) option for faster approximate posteriors.

A built‑in global sensitivity analysis module uses Sobol indices (first‑order and total‑order) computed via quasi‑Monte Carlo sampling and the SALib package. This helps identify influential parameters, reduce dimensionality, and guide prior specification, especially for high‑dimensional problems.

Sampling is supported through two pathways: (1) traditional MCMC, with automatic convergence diagnostics (effective sample size, R̂, trace plots) and (2) VI, with ELBO monitoring. Posterior predictive checks, residual diagnostics, and diagnostic plotting utilities are integrated, allowing users to assess model fit and calibration quality without leaving the Python environment.

Beyond the software, the authors compile a set of practical recommendations derived from applied experience and recent literature:

• Scale and normalize parameters before inference to improve sampler stability.
• Choose priors that reflect physical knowledge and respect the scale of the data.
• When experimental error variance is unknown, treat it as a parameter with a weakly informative prior.
• Use Sobol sensitivity results to prune non‑influential parameters before calibration.
• Monitor sampler convergence with multiple chains, target R̂ < 1.1 and ESS > 200 per parameter.
• Perform posterior predictive validation and compare against held‑out data.

The paper concludes with three illustrative examples: (i) a simple single‑output linear model, (ii) a multi‑output nonlinear finite‑element simulation, and (iii) a hybrid case where a surrogate, discrepancy, and experimental error are all active. Each example walks the reader through data preparation, model definition, sensitivity analysis, calibration execution, and post‑calibration diagnostics, showcasing ACBICI’s API and the types of outputs (posterior samples, predictive intervals, sensitivity rankings) that can be obtained.

Overall, the work bridges the gap between Bayesian calibration theory and practical, scalable implementation for complex, data‑scarce engineering models. By delivering a unified, extensible Python package together with detailed methodological guidance, it equips researchers and practitioners with the tools needed for rigorous, interpretable, and computationally efficient calibration of modern scientific simulations.