Solved by CDhere in post #3 
              
              
                Thank you for the reference! Yes this is super helpful. It worked for me to add these two lines: 
torch.backends.cudnn.benchmark = False
torch.set_deterministic(True)

So just to summarize, there are several sources of randomness when using Pytorch: 


Application-side randomness: this happens at ou…