Hi, I run below code on CPU (there is GPU available on this machine too):
import torch
x = torch.tensor([
        [1, 1, 1, 1, 1, 1, 1, 1,],
        [0.06770703196525574, 1, 1, 1, 1, 1, 1, 1,],
    ])
y = torch.nn.Sigmoid()(x)
print(y[1,0].data.cpu().numpy().tolist())
y = torch.nn.Sigmoid()(x[:,:-1])
print(y[1,0].data.cpu().numpy().tolist())

I would expect that this code will print two identical values, but instead it prints out:
0.5169203281402588
0.516920268535614

Interesting, if I run this code on GPU of this machine, I will see twice the same value “0.5169203281402588”
I am wondering what’s exactly going on here, and whether this behavior is expected? Only one way I would try to explain it is that depending on the size of matrix, my CPU runs somehow reuse(?) GPU for performance benefits, and then GPU has different floating point precision than the CPU causing the numbers to be different.