Machine Learning, Gradient Descent, Data Visualization, Data Science.
rescaling adjustment (1b), the ball would have moved mostly vertically downwards; with the adjustment (1d), it moves diagonally. AdaGrad (white) vs. gradient descent (cyan) on a terrain with a saddle point. The learning rate of AdaGrad is set to be higher than that of gradient descent, but the