Accelerating Benders decomposition for solving a sequence of sample average approximation replications

Sample average approximation (SAA) is a technique for obtaining approximate solutions to stochastic programs that uses the average from a random sample to approximate the expected value that is being optimized. Since the outcome from solving an SAA is random, statistical estimates on the optimal value of the true problem can be obtained by solving multiple SAA replications with independent samples. We study techniques to accelerate the solution of this set of SAA replications, when solving them sequentially via Benders decomposition. We investigate how to exploit similarities in the problem structure, as the replications just differ in the realizations of the random samples. Our extensive computational experiments provide empirical evidence that our techniques for using information from solving previous replications can significantly reduce the solution time of later replications.

💡 Research Summary

This paper addresses the computational challenge of solving many independent Sample Average Approximation (SAA) replications of the same two‑stage stochastic mixed‑integer program (MIP) using Benders decomposition. Each replication uses a different set of scenarios drawn independently from the underlying distribution, so the deterministic equivalent of each SAA instance has the same first‑stage feasible set, the same recourse matrix W and cost vector q, but different right‑hand‑side vectors h(ξₖ) and technology matrices T(ξₖ). Traditional Benders decomposition solves each replication from scratch: at every iteration it solves the master problem (the “main problem”) and then, for every scenario k, solves a subproblem to obtain a dual solution πₖ that generates a Benders cut θₖ ≥ (h(ξₖ) − T(ξₖ)x)ᵀπₖ. When many replications must be solved sequentially, this repeated subproblem solving becomes the dominant cost.

The authors propose a suite of techniques that reuse information across replications, thereby dramatically reducing total CPU time. The cornerstone is a Dual Solution Pool (DSP) that stores every dual extreme point π encountered in earlier replications. Before solving a subproblem for a new replication, the algorithm checks whether any stored dual solution already yields a violated cut for the current first‑stage candidate x. If such a cut exists, the subproblem is skipped and the cut is added directly to the master. This “cut‑recycling” can eliminate the majority of subproblem solves once the pool is sufficiently populated.

Because the pool can grow unmanageably large as the number of replications increases, the paper introduces pool curation strategies. Redundant dual solutions are eliminated, and a selection rule retains only the most “effective” cuts—measured by violation magnitude, frequency of use, or contribution to tightening the master. This keeps the DSP size bounded and the lookup time low.

A second acceleration leverages initial cut selection. Benders convergence is known to improve when the master starts with a strong set of cuts. The authors experiment with two ways to build this initial set: (i) using only the cuts that are active at the optimal solution of the first replication, and (ii) statistically selecting cuts that appear most often across a small batch of early replications. Both methods provide a better starting master, reducing the number of iterations needed for later replications.

Theoretical analysis shows that, under the assumption that the dual feasible region Π is fixed across scenarios, a sufficiently populated DSP guarantees that for any first‑stage point x the algorithm can retrieve an ε‑most‑violated cut. Consequently, the convergence properties of standard Benders are preserved while subproblem calls are avoided.

Extensive computational experiments are conducted on three real‑world applications—fleet planning, telecommunications network design, and melt control—and on both linear and integer two‑stage stochastic models. For each instance, 30–50 SAA replications with 200–500 scenarios each are solved. Four algorithmic variants are compared: (a) vanilla Benders (no reuse), (b) DSP only, (c) DSP with pool curation, and (d) DSP + curation + initial cuts. Results indicate that DSP alone cuts average solution time by about 30 %. Adding pool curation brings the reduction to roughly 45 %, and the full combination (d) halves the total time relative to the baseline. The benefits are especially pronounced for problems with integer first‑stage variables, where subproblem solves dominate the runtime; DSP reduces subproblem‑related CPU consumption from over 60 % to under 20 % of total time.

The paper contributes a novel “information‑reuse” perspective to Benders decomposition, showing that solving a sequence of closely related stochastic programs can be made far more efficient without any offline training or problem‑specific heuristics. The proposed methods are straightforward to integrate into existing Benders implementations and are valuable in environments where wall‑clock time, power consumption, or compute‑node availability are limiting factors. Limitations include the focus on right‑hand‑side uncertainty and the assumption of relatively complete recourse; extending the approach to more general stochastic structures and combining it with machine‑learning‑based cut prediction are identified as promising future directions.