Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
expressed in the following pictures: Left — SGD without momentum, right— SGD with momentum. (Source: Genevieve B. Orr) Conclusion I hope this post gave you some intuition on how and why SGD with momentum works. It’s actually one of the most popular optimization algorithms in deep learning and used