Deep Learning, Optimization, Gradient Descent, Momentum, Learning Parameters.
still stuck at an error of 0.36. Can we do something to reduce the oscillations/U-turns? Yes, Nesterov Accelerated Gradient Descent helps us do just that. Nesterov Accelerated Gradient Descent The intuition behind NAG can be put into a single phrase: Look ahead before you leap! The NAG Update Rule