Hi Thomas!



 tjak:

It seems as if the internal accumulator is only float16. It works with float64, or without using CUDA.
Cannot reproduce on Ubuntu machine.


FYI, I also cannot reproduce this on Ubuntu using a
“GeForce GTX 1050 Ti”.
>>> import torch
>>> dtype = torch.float32
>>> A = torch.tensor([[1.]], dtype=dtype).cuda()
>>> B = torch.tensor([[1.0001]], dtype=dtype).cuda()
>>> test1 = torch.matmul(A, B)
>>> A = torch.tensor([1.], dtype=dtype).cuda()
>>> B = torch.tensor([1.0001], dtype=dtype).cuda()
>>> test2 = torch.matmul(A, B)
>>> print(test1)
tensor([[1.0001]], device='cuda:0')
>>> print(test2)
tensor(1.0001, device='cuda:0')
>>> print(torch.version.cuda)
10.2
>>> print(torch.__version__)
1.9.0
>>> print (torch.cuda.get_device_name (0))
GeForce GTX 1050 Ti

$ uname -a
Linux server 5.11.0-38-generic #42~20.04.1-Ubuntu SMP Tue Sep 28 20:41:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Best.
K. Frank