As of pytorch 4.1 this is not the case anymore. relu(NaN) == NaN
In [1]: import torch

In [2]: x = torch.ones(1).float()+float('NaN')

In [3]: x
Out[3]: tensor([    nan])

In [4]: x.relu()
Out[4]: tensor([    nan])

I’ve previously happily (ab)used the previous behaviour. Is there a suggested new method on How to set ‘nan’ in Tensor to 0 14?
I assume the suggested method 14 my_tensor[torch.isnan(my_tensor)] = 0. will cause problems for GPU and have a high memory cost. Is there some other method?