Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
u-turns). Furthermore, both of them are faster than stochastic gradient descent (after 60 steps, stochastic gradient descent [black - left figure] still exhibits a very high error whereas NAG and momentum are close to convergence). And, of course, we can also have the mini-batch versions of