Machine Learning, Deep Learning, Gradient Descent, Neural Networks, Learning Rate.
decrease learning rates by steps: to simply use one learning rate for the first few iterations, then drop to another learning rate for the next few iterations, then drop the learning rate further for the next few iterations. Another variant would be to linearly decrease the learning rate with each