Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
descent update rule, we also included the history component vₜ it stores all the previous gradient movements till this time t. Exponential Weighted Average At every time step or every update instead of just moving by the value of the current gradient like the Vanilla GD. In Momentum GD, we are