Machine Learning, Gradient Descent, Stochastic, Optimization, TensorFlow.
SGD with Momentum, a “momentum” entity is introduced, which is a running mean of the accumulated gradients in the past. SGD with momentum reaches the minmum quickly. SGD with Momentum Visualization But, if not careful with the momentum and train longer, you can overshoot and go to undesired areas