Global Convergence to Nash Equilibrium in Nonconvex General-Sum Games under the $n$-Sided PL Condition

We consider the problem of finding a Nash equilibrium (NE) in a general-sum game, where player $i$’s objective is $f_i(x)=f_i(x_1,…,x_n)$, with $x_j\in\mathbb{R}^{d_j}$ denoting the strategy variables of player $j$. Our focus is on investigating first-order gradient-based algorithms and their variations, such as the block coordinate descent (BCD) algorithm, for tackling this problem. We introduce a set of conditions, called the $n$-sided PL condition, which extends the well-established gradient dominance condition a.k.a Polyak-Łojasiewicz (PL) condition and the concept of multi-convexity. This condition, satisfied by various classes of non-convex functions, allows us to analyze the convergence of various gradient descent (GD) algorithms. Moreover, our study delves into scenarios where the standard gradient descent methods fail to converge to NE. In such cases, we propose adapted variants of GD that converge towards NE and analyze their convergence rates. Finally, we evaluate the performance of the proposed algorithms through several experiments.

💡 Research Summary

This paper tackles the challenging problem of computing Nash equilibria (NE) in general‑sum games where each of the n players controls a possibly non‑convex objective function f_i(x₁,…,x_n). Classical convergence guarantees for multi‑agent learning rely on strong convexity, monotonicity, or the standard Polyak‑Łojasiewicz (PL) condition, which are rarely satisfied in realistic multi‑player settings with block‑structured variables. To bridge this gap, the authors introduce the n‑sided μ‑PL condition. For every player i and any fixed strategies of the other players x_{‑i}, the partial function f_i(·,x_{‑i}) satisfies a PL inequality with constant μ_i>0: ‖∇i f_i(x)‖² ≥ 2μ_i ( f_i(x_i,x{‑i}) – min_{y_i} f_i(y_i,x_{‑i}) ). Thus each block enjoys a gradient‑dominance property independently, while no global convexity is required. This condition is strictly weaker than multi‑strong convexity and extends the two‑sided PL condition previously used for min‑max games.

A central theoretical contribution is Lemma 2.5, which shows that under the n‑sided PL condition the set of partial stationary points (where each player’s own gradient vanishes) coincides exactly with the set of Nash equilibria. Consequently, any algorithm that drives each block’s gradient to zero automatically finds an NE, eliminating the usual gap between stationary points and equilibrium in non‑convex games.

The paper then studies Block Coordinate Descent (BCD) and its stochastic variant Random BCD (R‑BCD). In R‑BCD, at each iteration a player i is sampled uniformly, and its strategy is updated by a simple gradient step: x_i ← x_i – α ∇_i f_i(x). When the iterates remain bounded, the authors prove that the expected suboptimality decays as O(1/t), guaranteeing convergence to the NE set in the limit. However, this rate depends on the Lipschitz constants L_i of the partial gradients and can be slow in practice.

To accelerate convergence, the authors propose several enhanced BCD schemes:

1. Block‑wise step‑size selection (α_i = 1/L_i) to normalize the effect of heterogeneous smoothness.
2. Weighted updates that incorporate the PL constants μ_i, effectively scaling each block’s descent direction.
3. Nesterov‑type momentum applied to the block updates, yielding an accelerated variant with provable linear convergence under an additional local PL assumption (i.e., the PL inequality holds in a neighbourhood of the solution).

Under the global n‑sided PL together with a uniform bound μ = min_i μ_i and L = max_i L_i, and with a step size α ≤ 1/L, the enhanced algorithms achieve linear convergence: ‖x^t – x*‖ ≤ (1 – μ/L)^t ‖x⁰ – x*‖, where x* denotes any Nash equilibrium. The analysis carefully handles the asynchronous nature of block updates and leverages the equivalence between stationary points and NE.

The paper validates the theoretical framework on several representative problems:

• n‑player Linear‑Quadratic (LQ) games, where each player’s cost is quadratic in its own state and linear in the others’. By assuming positive‑definite self‑cost matrices, the LQ game satisfies the n‑sided PL condition.
• Linear residual networks, a class of deep linear models whose layer‑wise losses obey PL inequalities.
• Potential games with non‑convex potentials. The authors illustrate two 2‑player potentials (Figures 1) that are 2‑sided PL yet possess multiple NE with different objective values, highlighting that uniqueness is not guaranteed under the n‑sided PL condition.

Empirical experiments compare the basic R‑BCD, cyclic BCD, and the accelerated variants on synthetic potential functions f^{(3)} and f^{(4)}. The log‑scale convergence plots (Figure 2) demonstrate that f^{(4)} exhibits clear linear decay of the objective, confirming the theoretical rate, while f^{(3)} shows sub‑linear behavior. Additional tests on the n‑player LQ game reveal that the proposed algorithms converge 2–3× faster than standard Gradient Descent‑Ascent (GDA) and are less sensitive to step‑size tuning.

Significance and Limitations: The n‑sided PL condition provides a unifying structural assumption for a broad class of multi‑player, block‑structured non‑convex games, enabling global convergence guarantees for simple first‑order methods. By proving the equivalence of stationary points and Nash equilibria, the work simplifies algorithmic design: any block‑wise gradient descent that drives partial gradients to zero suffices. However, verifying the n‑sided PL condition in complex models (e.g., deep neural networks) may be non‑trivial, and the required constants μ_i, L_i can be difficult to estimate. Moreover, the analysis assumes centralized access to all gradients; extending the results to decentralized or communication‑constrained settings remains an open direction.

In conclusion, the paper makes three key contributions: (1) introducing the n‑sided PL condition as a novel, weaker-than‑convexity assumption for general‑sum games; (2) establishing that under this condition, block‑coordinate descent converges globally to Nash equilibria, with linear rates achievable under additional local PL assumptions; and (3) demonstrating the practical relevance of the theory through experiments on potential games, LQ games, and linear residual networks. This work opens new avenues for rigorous analysis of multi‑agent learning algorithms in non‑convex environments.