Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods

Recent research has indicated a substantial rise in interest in understanding Nesterov’s accelerated gradient methods via their continuous-time models. However, most existing studies focus on specific classes of Nesterov’s methods, which hinders the attainment of an in-depth understanding and a unified perspective. To address this deficit, we present generalized continuous-time models that cover a broad range of Nesterov’s methods, including those previously studied under existing continuous-time frameworks. Our key contributions are as follows. First, we identify the convergence rates of the generalized models, eliminating the need to determine the convergence rate for any specific continuous-time model derived from them. Second, we show that six existing continuous-time models are special cases of our generalized models, thereby positioning our framework as a unifying tool for analyzing and understanding these models. Third, we design a restart scheme for Nesterov’s methods based on our generalized models and show that it ensures a monotonic decrease in objective function values. Owing to the broad applicability of our models, this scheme can be used to a broader class of Nesterov’s methods compared to the original restart scheme. Fourth, we uncover a connection between our generalized models and gradient flow in continuous time, showing that the accelerated convergence rates of our generalized models can be attributed to a time reparametrization in gradient flow. Numerical experiment results are provided to support our theoretical analyses and results.

💡 Research Summary

The paper presents a unified continuous‑time framework that captures a broad family of Nesterov’s accelerated gradient (NAG) methods. By introducing an auxiliary “growth” sequence Aₖ, the authors rewrite the classic three‑sequence NAG update (yₖ, xₖ, zₖ) in terms of Aₖ and its differences. This reformulation yields two discrete schemes—one for the convex case (μ = 0) and one for the strongly‑convex case (μ > 0)—which are then taken to the continuous limit (step size h → 0) to obtain generalized ordinary differential equations (ODEs).

The resulting ODE system has the form

\