Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
Momentum and NAG but just not in this post. Conclusion In this part of the learning parameters series, we looked at variations of the gradient descent algorithm that approximate the gradients updates — Stochastic Gradient Descent and Mini-Batch Gradient Descent. We looked at the key differences