Machine Learning, Deep Learning, Gradient Descent, Neural Networks, Learning Rate.
local minimum will result in a large change of loss: Image 6: Slight shift in loss function resulting in a large change in loss Suddenly, this local minimum is a terrible solution. On the other hand, if we had found a solution in this flatter trough: Image 7: Approaching a more stable local minimum