Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
Nesterov accelerated gradient Nesterov Momentum is a slightly different version of the momentum update that has recently been gaining popularity. In this version we’re first looking at a point where current momentum is pointing to and computing gradients from that point. It becomes much clearer