Machine Learning, Deep Learning, Stochastic Gradient, Data Science, Editors Pick.
the learning rate. Instead of it being based on constants and tuning by trial and error. You can use, for instance, an adaptive algorithm, which picks the learning rate based on the gradient from previous iterations. For example, Adam[6] Adadelta[7] use a decaying average of past gradients.