Continuized Nesterov Momentum Achieves the $O(arepsilon^{-7/4})$ Complexity without Additional Mechanisms

For first-order optimization of non-convex functions with Lipschitz continuous gradient and Hessian, the best known complexity for reaching an $\varepsilon$-approximation of a stationary point is $O(\varepsilon^{-7/4})$. Existing algorithms achieving this bound are based on momentum, but are always complemented with safeguard mechanisms, such as restarts or negative-curvature exploitation steps. Whether such mechanisms are fundamentally necessary has remained an open question. Leveraging the continuized method, we show that a Nesterov momentum algorithm with stochastic parameters alone achieves the same complexity in expectation. This result holds up to a multiplicative stochastic factor with unit expectation and a restriction to a subset of the realizations, both of which are independent of the objective function. We empirically verify that these constitute mild limitations.

💡 Research Summary

The paper addresses the long‑standing open question of whether additional safeguard mechanisms—such as restarts, negative‑curvature exploitation steps, or other auxiliary procedures—are fundamentally required for first‑order methods to achieve the optimal O(ε⁻⁷ᐟ⁴) iteration complexity when minimizing non‑convex functions whose gradient and Hessian are both Lipschitz continuous. Existing algorithms that attain this rate rely on Nesterov momentum (NM) as the core update but augment it with a “security check” that, upon failure, triggers an alternative mechanism. This paper shows that the security check and its associated fallback steps are not essential: a purely momentum‑based algorithm, when equipped with stochastic parameters derived from a continuized formulation, already reaches the optimal rate in expectation.

Key Contributions

1. Continuized Nesterov Dynamics – The authors introduce a stochastic differential equation (CNE) that mixes continuous‑time linear dynamics with Poisson‑driven gradient jumps occurring at random exponential times. By sampling the process at jump times, they obtain a discrete algorithm (CNA) that has the same structure as NM but with random momentum coefficients αₖ and βₖ that depend on the inter‑jump intervals ΔTₖ.

2. Lyapunov Analysis in Continuous Time – Using a carefully constructed Lyapunov function, Lemma 3.1 proves that, under the Lipschitz‑gradient assumption alone, the expected integral of a combination of the squared distance between the two state variables and the squared gradient norm is bounded by the initial function suboptimality. This yields an O(1/t) decay of the expected gradient norm in continuous time.

3. Transfer to Discrete Time – By exploiting the martingale property of the Poisson integral, the continuous‑time bound is transferred to the discrete iterates of CNA, resulting in Proposition 3.2, which recovers the classic O(ε⁻²) bound for finding an ε‑stationary point—exactly the rate of standard Nesterov momentum in the Lipschitz‑gradient setting.

4. Accelerated Rate with Lipschitz Hessian – Building on the heavy‑ball ODE analysis of