Thanks for this part of the code!
Could you try to save the state_dict of our model running on the 1080Ti machine and load it on the 2080Ti one?
If you also get nan values, we would have to locate the exact code where the nan is generated.
torch.autograd.detect_anomaly 5 might be helpful for this.