Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
learning in the vertical direction, dividing db with Sdbin update step will result in a smaller change in b. Hence, learning in the vertical direction will be less. Similarly, a small value of SdW will result in faster learning in the horizontal direction, thus making the algorithm faster. Adam