Machine Learning, Deep Learning, Stochastic Gradient, Data Science, Editors Pick.
back and forth several times. With big steps, Gradient Descent may never even reach the minimum and converge. One way to spot that alpha is too big is when the gradient is not decreasing at every step. Learning rate is too small When the learning rate is too small, you’re taking tiny steps at a