Deep Learning, Optimization, Gradient Descent, Momentum, Learning Parameters.
repeatedly being asked to move in the same direction then I should probably gain some confidence and start taking bigger steps in that direction. Just as a ball gains momentum while rolling down a slope. The Momentum Update Rule We accommodate the momentum concept in the gradient update rule as