Revisiting Nesterov’s acceleration via high-resolution differential equations

Nesterov’s accelerated gradient descent (NAG) stands as a landmark in the development of first-order optimization algorithms. Its acceleration mechanism, which originates from a gradient correction term, was elucidated through the high-resolution ordinary differential equation (ODE) framework, as referenced in [1]. This framework has been vital in demystifying the effectiveness of NAG. Moreover, it is worth noting that the construction of Lyapunov functions within this framework is methodical and principled. In this paper, we leverage this framework to conduct an in-depth analysis of the convergence properties of NAG for $$\mu $$ μ -strongly convex functions. First, we refine the proof of the gradient-correction scheme, streamlining the process with straightforward calculations akin to that in [2]. This also allows us to enlarge the step size to $$s=1/L$$ s = 1 / L with only slight modifications. Furthermore, our analysis via the implicit-velocity scheme reveals that its associated Lyapunov function is more succinct to construct, as it simplifies the structure and eases the computation of the iterative difference. This resulting simplicity, coupled with the optimal step size derived, indicates the superiority of the implicit velocity scheme over the gradient correction scheme within the high-resolution ODE framework.