Solved by dejanbatanjac in post #4 
              
              
                Hi @shirui-japina, 
There is actually a guy called Leslie N. Smith who created this paper. 
Based on this paper, some other guy created the learning rate finder. 
What, it does, it measures the loss for the different learning rates and plots the diagram as this one: 
[image] 
It shows up (empiricall…