Solved by tom in post #13 
              
              
                @ptrblck actually already mentioned what’s going on there: The BatchNorm statistics are updated when you run in train mode, so you have a different network. 
Note that you need to check the state dict or the buffers, not just the parameters. 
To abstract the situation: You have something where you g…