BISTRO -- A Bi-Fidelity Stochastic Gradient Framework using Trust-Regions for Optimization Under Uncertainty

Stochastic optimization of engineering systems is often infeasible due to repeated evaluations of a computationally expensive, high-fidelity simulation. Bi-fidelity methods mitigate this challenge by leveraging a cheaper, approximate model to accelerate convergence. Most existing bi-fidelity approaches, however, exploit either design-space curvature or random-space correlation, not both. We present BISTRO - a BI-fidelity Stochastic Trust-Region Optimizer for unconstrained optimization under uncertainty through a stochastic approximation procedure. This approach exploits the curvature information of a low-fidelity objective function to converge within a basin of a local minimum of the high-fidelity model where low-fidelity curvature information is no longer valuable. The method then switches to a variance-reduced stochastic gradient descent procedure. We provide convergence guarantees in expectation under certain regularity assumptions and ensure the best-case $\mathcal{O}(1/n)$ convergence rate for stochastic optimization. On benchmark problems and a 20-dimensional space shuttle reentry case, BISTRO converges faster than adaptive sampling and variance reduction procedures and cuts computational expense by up to 29x.

💡 Research Summary

The paper “BISTRO – A Bi-Fidelity Stochastic Gradient Framework using Trust-Regions for Optimization Under Uncertainty” addresses the computational bottleneck of simulation-based stochastic optimization, where repeated evaluations of a high-fidelity, computationally expensive model are often prohibitive. The authors propose a novel algorithm named BISTRO (BI-fidelity Stochastic Trust-Region Optimizer) that uniquely leverages both design-space curvature and random-space correlation information across model fidelities to accelerate convergence.

The core innovation lies in a two-phase hybrid approach. In the first phase, BISTRO employs a bi-fidelity trust-region method. It uses the curvature information from a cheaper, low-fidelity model to propose large, informed steps, rapidly navigating the design space to reach the basin of attraction of a local minimum for the high-fidelity objective. The low-fidelity model is corrected to match the high-fidelity function value and gradient at the trust-region center, ensuring local consistency. This phase is highly effective when far from the optimum, where geometric information is more valuable than precise statistical estimates.

Once the algorithm converges to a region where the low-fidelity model’s curvature is no longer a reliable guide (typically near the high-fidelity optimum), it switches to the second phase: a variance-reduced stochastic gradient descent procedure. This phase utilizes control variates, specifically a multi-level Monte Carlo estimator, to drastically reduce the variance of the stochastic gradient estimates for the high-fidelity objective. This allows for efficient asymptotic convergence with the optimal O(1/n) rate under standard regularity conditions.

Theoretical analysis provides convergence guarantees in expectation to a stationary point of the high-fidelity problem. The paper also establishes conditions under which the variance-reduced phase converges faster than a standard single-fidelity stochastic gradient descent, formalizing the benefit of the low-fidelity model in reducing estimator variance. A key practical advantage of BISTRO over some adaptive sampling trust-region methods is that it does not asymptotically increase the per-iteration sample budget, maintaining computational efficiency.

Numerical experiments on benchmark problems, culminating in a complex 20-dimensional space shuttle reentry optimization case, demonstrate the algorithm’s efficacy. BISTRO significantly outperforms both adaptive sampling techniques and variance-reduction-only methods. It achieves up to a 29-fold reduction in the number of required high-fidelity simulations to reach a given solution accuracy, showcasing its potential for making high-dimensional, uncertainty-aware engineering design optimization tractable. The framework, while presented in a bi-fidelity context, is naturally extensible to ensembles of multiple low-fidelity models for further variance reduction.