Deep Learning, Optimization, Gradient Descent, Momentum, Learning Parameters.
Learning Parameters Learning Parameters, Part 2: Momentum-Based & Nesterov Accelerated Gradient Descent Let’s look at two simple, yet very useful variants of gradient descent. In this post, we look at how the gentle-surface limitation of Gradient Descent can be overcome using the concept of momentum