Hi @ptrblck!



 ptrblck:

Update: Fix is incoming. The root cause seems to be the missing accumulation dtype for the CPU path.
CC @KFrank @Dominika_Basaj


Just to confirm, I see that the fix is in in the latest nighty.  Here is the
result of my above-posted test script:
1.9.0.dev20210416
nBatch = 128
nClass = 2
height = 768
width = 768
big, float:
mean: tensor(0.9028)
none: tensor(0.9028)
sum: tensor(0.9028)
big, cuda:
mean: tensor(0.9028, device='cuda:0')
none: tensor(0.9028, device='cuda:0')
sum: tensor(0.9028, device='cuda:0')
big, double:
mean: tensor(0.9028, dtype=torch.float64)
none: tensor(0.9028, dtype=torch.float64)
sum: tensor(0.9028, dtype=torch.float64)
nBatch = 8
nClass = 2
height = 16
width = 16
small, float:
mean: tensor(0.9148)
none: tensor(0.9148)
sum: tensor(0.9148)
small, cuda:
mean: tensor(0.9148, device='cuda:0')
none: tensor(0.9148, device='cuda:0')
sum: tensor(0.9148, device='cuda:0')
small, double:
mean: tensor(0.9148, dtype=torch.float64)
none: tensor(0.9148, dtype=torch.float64)
sum: tensor(0.9148, dtype=torch.float64)

Best.
K. Frank