Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
Momentum The underlying idea of gradient descent with momentum is to calculate the exponential weighted average of gradients and use them to update weights. Suppose we have a cost function whose contours look like this: The red dot is the global minima, and we want to reach that point. Using