Machine Learning, Deep Learning, Gradient Descent, Neural Networks, Learning Rate.
rates for several iterations in the beginning to quickly approach a local minimum, then gradually decrease the learning rate as we get closer to the minimum, ending with several small learning rate iterations. However, we may find ourselves in a local minimum where a small change in weights would