Solved by albanD in post #2 
              
              
                Hi, 

The thing is that in your case, you have a ReLU between the Linear and Batchnorm. So that statement may not be true for your model. 
I think that statement comes from the fact that the batchnorm will center the values. So a bias is useless in the previous layer as it will just be cancelled by …