Information-Theoretic Multi-Model Fusion for Target-Oriented Adaptive Sampling in Materials Design

Target-oriented discovery under limited evaluation budgets requires making reliable progress in high-dimensional, heterogeneous design spaces where each new measurement is costly, whether experimental or high-fidelity simulation. We present an information-theoretic framework for target-oriented adaptive sampling that reframes optimization as trajectory discovery: instead of approximating the full response surface, the method maintains and refines a low-entropy information state that concentrates search on target-relevant directions. The approach couples data, model beliefs, and physics/structure priors through dimension-aware information budgeting, adaptive bootstrapped distillation over a heterogeneous surrogate reservoir, and structure-aware candidate manifold analysis with Kalman-inspired multi-model fusion to balance consensus-driven exploitation and disagreement-driven exploration. Evaluated under a single unified protocol without dataset-specific tuning, the framework improves sample efficiency and reliability across 14 single- and multi-objective materials design tasks spanning candidate pools from $600$ to $4 \times 10^6$ and feature dimensions from $10$ to $10^3$, typically reaching top-performing regions within 100 evaluations. Complementary 20-dimensional synthetic benchmarks (Ackley, Rastrigin, Schwefel) further demonstrate robustness to rugged and multimodal landscapes.

💡 Research Summary

The paper tackles the challenge of discovering optimal material properties when evaluations (experiments or high‑fidelity simulations) are extremely costly and the design space is high‑dimensional and heterogeneous. Rather than building an accurate surrogate of the entire response surface, the authors propose an information‑theoretic framework that treats optimization as a trajectory‑discovery problem: a low‑entropy information state is maintained and progressively refined to concentrate search on directions that are most relevant to the target.

The framework consists of four tightly coupled stages. First, dimension‑aware information budgeting estimates the intrinsic dimensionality of the data observed so far and aligns the remaining evaluation budget with the effective capacity of the surrogate models. This prevents over‑fitting to high‑dimensional noise and forces the models to focus on the low‑dimensional manifold where physically plausible solutions reside. Second, a bootstrapped model distillation step creates a heterogeneous reservoir of surrogates (linear, tree‑based, neural networks) each trained on different bootstrap resamples. The resulting “cognitive landscapes” provide complementary inductive biases: low‑capacity models capture global trends, while high‑capacity models resolve fine‑grained local variations. Third, a structure‑aware candidate manifold analysis groups candidates using nearest‑neighbour and partition‑based metrics, quantifying redundancy, local density, and feasible variation without requiring an explicit embedding. When no explicit candidate set is supplied, the framework samples from the cognitive landscapes and projects those points onto the estimated manifold to verify feasibility. Fourth, a Kalman‑inspired multi‑model fusion (KF) and its uncertainty‑weighted variant (rKF) combine model outputs with the manifold information. Out‑of‑bag diagnostics provide per‑model signal‑to‑noise ratios and indicate whether a model’s cognition is globally or locally oriented. The fusion mechanism dynamically switches between consensus‑driven exploitation (standard acquisition functions) and disagreement‑driven exploration (uncertainty‑weighted variance), thereby maximizing mutual information between the tripartite cognitive system (data, models, physics) and the design target.

The authors evaluate the method on fourteen real‑world materials design tasks covering single‑ and multi‑objective problems, candidate pools ranging from 600 to 4 × 10⁶, feature dimensions from 10 to 10³, and up to four target properties. Under a unified protocol—20 random low‑quality initial points, batches of 10 new evaluations per iteration, and a total budget of ≤ 100 evaluations—the framework consistently identifies top‑10 candidates, often within ten iterations. Compared to standard Gaussian‑process Bayesian optimization (including deep kernel variants), the proposed approach achieves 2–3× higher sample efficiency and exhibits far more stable convergence.

Synthetic benchmarks include 20‑dimensional Ackley, Rastrigin, and Schwefel functions, each representing a distinct difficulty: weak gradients, dense multimodality, and deceptive basins, respectively. The method rapidly localizes the narrow valley of Ackley, successfully disambiguates the many local minima of Rastrigin, and performs high‑gain barrier‑crossing jumps to reach the distant global optimum of Schwefel. Across 20 independent runs per function, the best‑so‑far objective improves monotonically, and the distance to the true optimum decays exponentially for Ackley and shows stepwise reductions for Rastrigin and Schwefel, confirming robustness to varied landscape complexities.

Key contributions are: (1) a principled dimension‑aware capacity control that aligns model complexity with data scarcity, (2) a bootstrapped multi‑model ensemble that captures multi‑scale information, (3) a structure‑aware candidate manifold layer that filters out redundant or infeasible regions, and (4) an active information‑fusion mechanism that arbitrates between exploitation and exploration based on real‑time reliability estimates. Limitations include the computational overhead of manifold analysis for very large candidate sets and the reliance on empirically chosen numbers of bootstrap models and hyper‑parameters. Future work may integrate graph‑neural‑network embeddings for scalable manifold estimation and automated portfolio optimization of surrogate models.

In summary, the paper reframes high‑dimensional, data‑scarce materials design from exhaustive surface modeling to low‑entropy trajectory discovery, delivering a versatile, sample‑efficient, and theoretically grounded adaptive sampling strategy that outperforms conventional Bayesian optimization across both real‑world and synthetic benchmarks.