Solved by ptrblck in post #2 
              
              
                You might have forgotten to call model_transfer.eval(), which will disable dropout and will use the running stats in batchnorm layers instead of the current batch stats. 
Let me know, if that’s the case or if you are still seeing shaky outputs.