Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
Learning Parameters  Let’s digress a bit from optimizers and talk about the stochastic versions of these algorithms. In part 2, we looked at two useful variants of gradient descent — Momentum-Based and Nesterov Accelerated Gradient Descent. In this post, we are going to look at stochastic versions