Adaptive Surrogate-Based Strategy for Accelerating Convergence Speed when Solving Expensive Unconstrained Multi-Objective Optimisation Problems

Multi-Objective Evolutionary Algorithms (MOEAs) have proven effective at solving Multi-Objective Optimisation Problems (MOOPs). However, their performance can be significantly hindered when applied to computationally intensive industrial problems. To address this limitation, we propose an adaptive surrogate modelling approach designed to accelerate the early-stage convergence speed of state-of-the-art MOEAs. This is important because it ensures that a solver can identify optimal or near-optimal solutions with relatively few fitness function evaluations, thereby saving both time and computational resources. Our method employs a two-loop architecture. The outer loop runs a (baseline) host MOEA which carries out true fitness evaluations. The inner loop contains an Adaptive Accelerator that leverages data-driven machine learning (ML) surrogate models to approximate fitness functions. Integrated with NSGA-II and MOEA/D, our approach was tested on 31 widely known benchmark problems and a real-world North Sea fish abundance modelling case study. The results demonstrate that by incorporating Gaussian Process Regression, one-dimensional Convolutional Neural Networks, and Random Forest Regression, our proposed approach significantly accelerates the convergence speed of MOEAs in the early phases of optimisation.

💡 Research Summary

The paper addresses the challenge of solving computationally intensive multi‑objective optimisation problems (CI‑MOOPs) by proposing an adaptive surrogate‑based acceleration framework that speeds up the early‑stage convergence of state‑of‑the‑art evolutionary multi‑objective algorithms (MOEAs). The authors adopt a two‑loop architecture: an outer loop runs a baseline MOEA (NSGA‑II or MOEA/D) that performs true fitness evaluations, while an inner loop hosts an “Adaptive Accelerator” that builds and queries data‑driven surrogate models. Training data for the surrogates are taken directly from the most recent generation, eliminating the need for large archives, clustering, or complex filtering.

Three regression models are investigated – Gaussian Process Regression (GPR), one‑dimensional Convolutional Neural Networks (1‑D CNN), and Random Forest Regression (RFR). GPR provides both predictions and uncertainty estimates, enabling a principled exploration‑exploitation balance. The 1‑D CNN captures local patterns in continuous design variables, offering rapid convergence even in higher‑dimensional spaces. RFR excels at modelling non‑linear relationships with limited data, delivering stable early‑stage predictions.

The surrogate is activated after the second generation, ensuring that enough true evaluations have been collected to train a useful model. An adaptive exit mechanism monitors surrogate performance; when the surrogate ceases to contribute improvement, it is automatically disabled to avoid unnecessary overhead. Within the inner loop, surrogate‑generated candidate solutions are selected via a simple random strategy and passed back to the outer loop for re‑evaluation with the true fitness function. This modular integration minimally disrupts the MOEA’s selection, crossover, and mutation operators.

Experimental validation includes 31 benchmark problems (DTLZ, WFG, ZDT families) and a real‑world case study of North Sea fish abundance modelling. Performance metrics such as Hypervolume and Inverted Generational Distance (IGD) are evaluated across early evaluation budgets (10‑30 % of total calls). Results show that the surrogate‑augmented variants achieve on average more than a two‑fold acceleration in convergence during the early phases, with especially pronounced gains on high‑dimensional problems (≥10 variables). In the fish abundance case, the method finds Pareto‑near‑optimal solutions with fewer than 1,000 true fitness evaluations, demonstrating practical feasibility for time‑critical industrial optimisation.

Key contributions are: (1) a plug‑and‑play Adaptive Accelerator that can incorporate any regression surrogate, allowing easy integration with existing MOEAs; (2) a lightweight training‑data generation scheme that uses only the immediate previous generation, removing archive management overhead; (3) an uncertainty‑driven adaptive termination criterion that prevents surrogate over‑use; and (4) extensive empirical evidence of early‑stage speed‑up across synthetic benchmarks and a real‑world application.

Future work suggested includes extending the framework to constrained multi‑objective problems, leveraging surrogate uncertainty for dynamic exploration‑exploitation strategies, and developing hybrid evaluation scheduling to further optimise computational resource allocation. The proposed approach thus offers a practical pathway to accelerate expensive multi‑objective optimisation in engineering and scientific domains.