Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
Loss (Left) & Large Steps towards Loss (Right) In short, in the gentle areas of the loss surface, the movement is very slow and in the steep areas the movement is fast and again at the gentle areas at the bottom, the movement is very slow. But Why? We have made a pretty interesting observation