Could you check, if the model output contains NaN values before calculating the loss and calling backward()?
Did you check the inputs, if they contain finite elements?
Also, could you rerun the script with torch.autograd.set_detect_anomaly(True) and post the stack trace here, please?
If I understand it correctly, you are getting these NaN values in the second iteration?
If so, are you able to reproduce this behavior with random inputs, so that we could debug it?