Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
larger and larger steps evidently leads to overshooting the objective. Despite the u-turns, it still converges faster than the vanilla gradient descent. Nesterov Accelerated Gradient Descent How can we avoid the overshooting of minima?. In Momentum based gradient descent we are making one movement