Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
direction and a faster learning in the horizontal direction which will help us to reach the global minima much faster. Let’s see how we can achieve it using momentum: On iteration t: Compute dW, dB on current mini-batch using momentum VdW = β * VdW + (1 - β) * dW Vdb = β * Vdb + (1 - β) * db Update