Could you try running your code with torch.autograd.detact_anomaly? Automatic differentiation package - torch.autograd — PyTorch 1.9.1 documentation 19
You could also check for an overflow when scaling your gradient? If you’re running with a low bit-wdith, scaling too high might cause an overflow which would create NaNs when backproping! Also, check for Infs as they can lead to NaNs under certain operations!