An information-matching approach to optimal experimental design and active learning

The efficacy of mathematical models heavily depends on the quality of the training data, yet collecting sufficient data is often expensive and challenging. Many modeling applications require inferring parameters only as a means to predict other quantities of interest (QoI). Because models often contain many unidentifiable (sloppy) parameters, QoIs often depend on a relatively small number of parameter combinations. Therefore, we introduce an information-matching criterion based on the Fisher Information Matrix to select the most informative training data from a candidate pool. This method ensures that the selected data contain sufficient information to learn only those parameters that are needed to constrain downstream QoIs. It is formulated as a convex optimization problem, making it scalable to large models and datasets. We demonstrate the effectiveness of this approach across various modeling problems in diverse scientific fields, including power systems and underwater acoustics. Finally, we use information-matching as a query function within an Active Learning loop for material science applications. In all these applications, we find that a relatively small set of optimal training data can provide the necessary information for achieving precise predictions. These results are encouraging for diverse future applications, particularly active learning in large machine learning models.

💡 Research Summary

The paper introduces an “information‑matching” framework for optimal experimental design (OED) and active learning (AL) that directly targets the precision of quantities of interest (QoIs) rather than the overall parameter uncertainty. Traditional OED criteria such as A‑optimality (trace minimization), D‑optimality (determinant maximization), and E‑optimality (maximizing the smallest eigenvalue) focus on reducing some global measure of the Fisher Information Matrix (FIM) of the parameters. However, many scientific models are “sloppy”: most parameters are unidentifiable, yet only a few linear combinations affect the QoIs. Consequently, minimizing parameter variance does not guarantee accurate QoI predictions.

The authors formulate two FIMs: one for the training data set (\mathcal{D}f) and one for the QoIs. For a candidate pool of M data points ({x_m}) with weights (w_m), the training‑data FIM is
(I(\theta)=\sum{m=1}^M w_m J_f(\theta;x_m)^{!T} J_f(\theta;x_m)),
where (J_f) is the Jacobian of the model (f(\theta;x)). The QoI‑target FIM is defined from a desired covariance (\Sigma) as
(J(\theta)=J_g(\theta)^{!T}\Sigma^{-1}J_g(\theta)),
with (J_g) the Jacobian of the QoI mapping (g(\theta;y)). The central constraint is the matrix inequality (I \succeq J), meaning the information supplied by the selected training data must dominate the information required to achieve the QoI precision.

To obtain a sparse selection of data points, the authors solve the convex optimization problem
\