Why different results when multiplying in CPU than in GPU?
I am not sure if this is Pytorch related…apologies if not. <SCODE>In [1]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
import numpy as np
import torch

a = torch.from_numpy(np.random.rand(5000,100000).astype(np.float32))
b = torch.from_numpy(np.random.rand(5000,100000).astype(np.float32))

c = a.cuda()
d = b.cuda()

print(a.dot(b))
print(c.dot(d))
:<EOF>
124996952.0
124997016.0
<ECODE>
tunante
Hi, (I have also run into this e.g. when computing the mean over all images on a dataset with float32.)
tom
Thanks, you are right about the float64. The number of different digits is similar (depends on the experiment), but they are way more closer numbers. <SCODE>import numpy as np
import torch

a = torch.from_numpy(np.random.rand(5000,100000).astype(np.float64))
b = torch.from_numpy(np.random.rand(5000,100000).astype(np.float64))

c = a.cuda()
d = b.cuda()

print(a.dot(b))
print(c.dot(d))::::::::::
:<EOF>
125000868.65247717
125000868.65247723
<ECODE>
tunante
CPU: <SCODE>(((a + b) + c) + d)
<ECODE> GPU: <SCODE>((a+b) + (c+d))
<ECODE>
kmichaelkills
smth
tunante
it is because GPUs have thousands of cores, and doing a map-reduce style computation best exploits the parallelism of GPUs
smth