Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
between NAG(Red curve) & Momentum(Black curve) We can see that all the oscillations made by the Nesterov Accelerated Gradient are much smaller than that of Momentum based Gradient Descent. Looking ahead helps NAG in correcting its course quicker than Momentum based Gradient Descent. Hence the