I’m doing some experiments with MultiLabelClassification and I was trying to calculate the mean of the loss manually and found out, that there is a big difference to the BCEWithLogitsLoss-Mean and I don’t get why that’s the case.
Here is my code:
import math, torch, torch.nn as nn
pre = torch.tensor([0.133, 0.145, 0.692, 0.030])
des = torch.tensor([0., 0., 1., 0.])
torch.sigmoid(pre)
--> tensor([0.5332, 0.5362, 0.6664, 0.5075])

l1 = -(1*(math.log(0.5332))+(1-0)*math.log(1-0.5332))
--> 1.3907130692473197

l2 = -(1*(math.log(0.5362))+(1-0)*math.log(1-0.5362))
--> 1.391549907340984

l3 = -(1*(math.log(0.6664))+(1-1)*math.log(1-0.6664))
--> 0.4058651881295041

l4 = -(1*(math.log(0.5075))+(1-0)*math.log(1-0.5075))
--> 1.3865193864361882

1/4*(l1+l2+l3+l4)
--> 1.143661887788499


And this is the pytorch-implementation:
import math, torch, torch.nn as nn
criterion = nn.BCEWithLogitsLoss(reduction='mean')
pre = torch.tensor([0.133, 0.145, 0.692, 0.030])
des = torch.tensor([0., 0., 1., 0.])
criterion(pre, des)
--> tensor(0.6611)

If I set the reduction to ‘none’ only l3 (Loss 3) hast the same value:
tensor([0.7619, 0.7683, 0.4058, 0.7083])

Could somebody please explain to me, why that’s the case? 
Thanks!