Hi Alban,
Thank you so much for your help.
My understanding now is that because MSELoss will use its activation for back propagation, once NaN appeared then it will be accounted for regardless of whatever operation I use afterwards.
I used the following code instead to resolve the issue.
    # loss = torch.nn.MSELoss(reduction='none')(out, y)
    nan = torch.isnan(y)
    y = torch.where(nan, torch.tensor(0.0), y)
    out = torch.where(nan, torch.tensor(0.0), out)
    loss = (out - y) ** 2

Thanks,
Seongmin