Solved by albanD in post #2 
              
              
                Hi, 
The difference is that in the resnet there is a batchnorm between the conv and the relu. 
The conv operation need it’s output to be able to compute the backward pass. The batchnorm operations does not need it’s output to compute the backward pass. 
So the operation that comes just after batchno…