Machine Learning, Deep Learning, Stochastic Gradient, Data Science, Editors Pick.
the problem and the dataset at hand. The simplest algorithm is to pick a constant, a small enough number, say 10^-3. You start using it and gradually increase it or decrease it depending on how fast or slow gradient descent finds the minimum. But, ideally you’d want to have an automated way to pick