May be when one of your models intercept an exploding gradient point, then some of your weights may overflow, which eventually may corrupt your values as nan. Could you please

try it with lowering weight initializations and learning rate?
or
Normalize your data values?
https://stackoverflow.com/questions/51033066/pytorch-loss-inf-nan 8