JuanFMontesinos:

If you want to do your own optimization you should zeroe the gradients each step.
lastly, I don’t know whether this function is backpropagable or not torch.nan_to_num(X_hat - pts, nan=0.0)    But I cannot imagine why you would have NaNs


I see your point. Sometimes my numpy array pts have missing rows; my toy exercise is to try and recover those points. It looks something like:
pts 
>>>array([[1327.769     ,  922.555     ,   86.56067961],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [1327.34660274,  921.29142466,   78.19481833],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [          nan,           nan,           nan],
       [1316.07260274,  920.69542466,   85.07831347]])



I’m trying to set the loss on those missing rows to zero (ie. torch.nan_to_num(X_hat - pts, nan=0.0), while taking the real L2 norm on those non-missing values. Is there a function that allows PyTorch to consider nan values in its loss as 0?