Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
when you look at the picture. Source (Stanford CS231n class) Nesterov momentum can be define with the following formulas: Why momentum works In this section I’d like to talk a little bit about why momentum most of the times will be better than classic SGD. With Stochastic Gradient Descent we don’t