Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
between them, python code implementations of both the methods and also illustrated their convergence graphically on a toy problem. We also visualized stochastic versions of Momentum and NAG. In the next post, we will discuss a few useful tips for adjusting the learning rate and momentum related