Hi Sam!



 Sam_Lerman:

I am running two Conv2d  layers on a tensor of nans and getting -infs as output.


This is strange, and not something I would expect to happen.  After all,
“nan” is the “One Floating-Point Number to rule them all.”
“nan” should infect all arithmetic and turn the results into “nan”:
>>> import torch
>>> torch.__version__
'1.9.0'
>>> tnan = torch.tensor ([float ('nan')])
>>> tinf = torch.tensor ([float ('inf')])
>>> tnan
tensor([nan])
>>> tnan + tinf
tensor([nan])
>>> tnan * tinf
tensor([nan])
>>> tnan / 0.0
tensor([nan])

As expected, when I feed “nan” into a Conv2d, I get “nan” out:
>>> conv = torch.nn.Conv2d (1, 1, 3)
>>> t = torch.randn (1, 1, 10, 10)
>>> t[0, 0, 4, 4] = tnan
>>> conv (t)
tensor([[[[ 0.0393,  0.0779,  0.0731,  0.8591, -0.8162,  0.6578, -0.8222,
            0.8873],
          [-1.3210,  1.4260,  0.0526, -0.5711,  1.2245,  0.7201, -0.3848,
           -0.2762],
          [ 0.1857, -0.7266,     nan,     nan,     nan,  0.8714,  0.4864,
            0.2397],
          [-0.7764,  1.5293,     nan,     nan,     nan, -0.0663,  0.2233,
           -0.5896],
          [ 0.2750, -0.0982,     nan,     nan,     nan, -0.2574, -0.1529,
            0.5295],
          [ 0.7697,  1.0994, -0.3693, -0.5683, -0.4822,  0.9385, -0.7202,
            0.7361],
          [-1.3883,  0.6983,  0.3545,  0.8573, -0.0080, -0.2240,  0.2517,
           -0.4848],
          [-0.5190,  0.1873,  0.3782,  0.3108, -0.1297, -0.5012,  1.1124,
           -0.7626]]]], grad_fn=<ThnnConv2DBackward>)

Can you reproduce this with short, self-contained, runnable script?
Best.
K. Frank