Solved by JuanFMontesinos in post #2 
              
              
                It doesn’t affect the performance if you properly do it. Models are supposed to be robust to randomness. 
The fact is that you also have to save optimizer state_dict as some of them has running parameters which affects loss, eg, Adam. 
If you don’t do so, you will observe a peak in the loss (which s…