Distributed Computing for Huge-Scale Aggregative Convex Programming

Concerning huge-scale aggregative convex programming of a linear objective subject to the affine constraints of equality and inequality and the quadratic constraints of inequality, convex and aggregatively computable, an algorithm is developed for its distributed computing. The consensus with single common variable is used to partition the constraints into multi-consensus blocks, and the subblocks of each consensus block are employed to partition the primal variables into multiple sets of disjoint subvectors. The global consensus constraints of equality and the original constraints are converted into the extended constraints of equality via slack variables to help resolve feasibility and initialization of the algorithm. The augmented Lagrangian, the block-coordinate Gauss-Seidel method, the proximal point method with double proximal terms or single, and ADMM are used to update the primal and slack variables; descent models with built-in bounds are used to update the dual. Convergence of the algorithm to optimal solutions is argued and the rate of convergence, $O(1/k^{1/2})$ is estimated, under the feasibility supposed. Issues requiring further explorations are listed.

💡 Research Summary

The paper addresses the challenge of solving huge‑scale aggregative convex programming problems that feature a linear objective together with affine equality and inequality constraints as well as convex quadratic inequality constraints. The authors propose a distributed algorithm that leverages a single global consensus variable to decompose the original problem into multiple consensus blocks. Each block contains a subset of the original constraints, and the primal variables are further partitioned into disjoint subvectors so that updates can be performed locally and in parallel.

To guarantee feasibility and to simplify initialization, the original equality constraints and the block‑wise equality constraints are first relaxed into pairwise inequalities (X_i − Z ≤ 0 and Z − X_i ≤ 0) and then transformed into extended equality constraints by introducing slack variables for every inequality. These slack variables are bounded by user‑defined upper limits, which serve both as feasibility guards and as a means to keep the augmented system well‑conditioned.

The core of the algorithm is built on an augmented Lagrangian formulation. For each consensus block i, a local augmented Lagrangian L_i is defined, incorporating the linear objective, the consensus terms, the original constraints, and quadratic penalty terms weighted by a positive penalty parameter ρ_i. The primal variables X_i,l (the l‑th subvector of block i) are updated using a block‑coordinate Gauss‑Seidel scheme combined with a double‑proximal point method: a ‖·‖_1 proximal term with coefficient σ_1 and a ‖·‖_2^2 proximal term with coefficient σ_2 are added to each subproblem. This double‑proximal structure improves numerical stability and ensures that each subproblem remains strongly convex.

Slack variables (p_XY_i, n_XY_i, F_Y_i, G_Y_i, p_HY_i, n_HY_i) and their associated dual variables (the Lagrange multipliers) are updated by solving simple quadratic subproblems that also include individual proximal terms (γ‑coefficients). The global consensus variable Z is updated by aggregating contributions from all blocks; the authors propose a sequential accumulation scheme that avoids a single massive reduction operation, thereby reducing communication overhead.

Dual updates are derived from first‑order optimality conditions of the convex subproblems, leading to inclusion relations that involve subgradients of indicator functions for the bound constraints. The authors prove a descent lemma (Lemma 1) showing that the augmented Lagrangian sequence {L_k} is non‑increasing and that the sum of the decrease terms D_k plus the proximal correction terms P_k is bounded below. From this they establish convergence of the primal‑dual iterates to a KKT point of the original problem, assuming the existence of a feasible solution. The convergence rate is shown to be O(1/√k), which, while slower than the classic O(1/k) rate of some ADMM variants, is justified by the presence of the double‑proximal terms and the extensive slack‑variable reformulation.

The paper also discusses practical aspects such as the selection of penalty parameters ρ_i, proximal coefficients σ and γ, and the upper bounds for slack variables. It highlights the trade‑off between the number of consensus blocks N, the number of subvector partitions M, and the resulting communication/computation load. Finally, the authors list several avenues for future work, including extensions to non‑linear objectives, asynchronous update schemes, large‑scale fluid‑dynamics simulations, and empirical scalability studies on high‑performance computing platforms.