Machine Learning, Deep Learning, Gradient Descent, Neural Networks, Learning Rate.
the local minimum: Image 13: “Restarting” doesn’t take us out of this local minimum Finally, we gradually decrease the learning rate again until we minimize the loss function, finding a stable solution. Image 14: Settling into this local minimum Each of these ‘periods’ is known as a cycle, and in