Machine Learning, Gradient Descent, Stochastic, Optimization, TensorFlow.
of gradients over time and adding that to the current gradient. This is in fact the idea behind momentum. In this post, we will be talking about Stochastic Gradient Decent (SGD) methods. Vanilla SGD Here’s what SGD looks like in a simple optimization problem SGD Optimization SGD with Momentum In