Machine Learning, Data Science, Artificial Intelligence, Programming, Towards Data Science.
initial learning rate of 0.001. To get better accuracy, we also used the cyclic learning rate approach when the learning rate oscillates between different values. As you can guess from this image, the learning rate was decreased around epoch 17. At the same time, there are two-loss peaks with