A Unifying Primal-Dual Proximal Framework for Distributed Nonconvex Optimization

We consider distributed nonconvex optimization over an undirected network, where each node privately possesses its local objective and communicates exclusively with its neighboring nodes, striving to collectively achieve a common optimal solution. To handle the nonconvexity of the objective, we linearize the augmented Lagrangian function and introduce a time-varying proximal term. This approach leads to a Unifying Primal-Dual Proximal (UPP) framework that unifies a variety of existing first-order and second-order methods. Building on this framework, we further derive two specialized realizations with different communication strategies, namely UPP-MC and UPP-SC. We prove that both UPP-MC and UPP-SC achieve stationary solutions for nonconvex smooth problems at a sublinear rate. Furthermore, under the additional Polyak-Łojasiewics (P-Ł) condition, UPP-MC is linearly convergent to the global optimum. These convergence results provide new or improved guarantees for many existing methods that can be viewed as specializations of UPP-MC or UPP-SC. To further optimize the mixing process, we incorporate Chebyshev acceleration into UPP-SC, resulting in UPP-SC-OPT, which attains an optimal communication complexity bound. Extensive experiments across diverse network topologies demonstrate that our proposed algorithms outperform state-of-the-art methods in both convergence speed and communication efficiency.

💡 Research Summary

**
This paper tackles the challenging problem of distributed nonconvex optimization over an undirected network, where each of the N agents holds a private smooth loss function f_i(x) and the goal is to minimize the global sum f(x)=∑_{i=1}^N f_i(x) while reaching consensus on the decision variable x. Classical consensus‑based methods such as ADMM, EXTRA, DIGing, and their nonconvex extensions typically rely on exact augmented‑Lagrangian (AL) minimization or gradient‑tracking schemes. However, directly applying these techniques to nonconvex objectives either yields weak convergence guarantees or incurs prohibitive communication costs, especially on sparse graphs where the Laplacian condition number γ is large.

The authors propose a Unifying Primal‑Dual Proximal (UPP) framework that blends three key ideas: (i) a first‑order (linear) approximation of the AL function at the current iterate, (ii) a time‑varying proximal matrix B_k that regularizes the primal step, and (iii) flexible dual ascent parameters θ and ρ. The primal update becomes a closed‑form expression

x^{k+1}=x^{k}−(B_k+ρH)^{-1}