Solved by LeviViana in post #2 
              
              
                You might find useful reading carefully the article. 
Batch norm enables training with larger learning rates, and this other article argues “…that the larger learning rate increases the implicit regularization of SGD, which improves generalization…”. 
I believe you could start reading these two arti…