Machine Learning, Artificial Intelligence, Neural Networks, Data Science, Towards Data Science.
dependence, the sweet spot is also spatially dependent — since certain locations on the loss surface may have extremely sharp or shallow gradients — which further complicates matters. So how do we choose a learning rate that will give us the best results? Clearly, we could just use a constant