Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks.
increases the speed of convergence In some cases, it can also help in overcoming the local minima Update steps in convergence using Momentum Optimizer | Source: Image by the author. One typical drawback of the Momentum Optimizer is that the algorithm tends to oscillate at the minima (like a ball