If the invalid values are created during the backward pass, you could add torch.autograd.set_detect_anomaly(True) at the beginning of your script, which should yield a stack trace pointing to the operation in the backward pass, which caused the NaNs.