Hi @shirui-japina I wrote a blog post which might provide a bit more insight on picking learning rates, hope it helps: http://www.bdhammel.com/learning-rates/ 122
TL;DR: Just use the learning rate finder.
For even more in-depth understanding, I strongly recommend giving this blog post a read:

  
      
      Myrtle – 24 Sep 18
  
  
    

How to Train Your ResNet 68

The introduction to a series of posts investigating how to train Residual networks efficiently on the CIFAR10 image classification dataset. By the fourth post, we can train to the 94% accuracy threshold of the DAWNBench competition in 79 seconds on a...


  
  
    
    
  
  


in particular, post #5 on hyperparameters