Hi, everybody, I met a strange problem when trying to define a weighted cross entropy myself.
My loss func is simple:
def weighted_BCE(sigmoid_x, targets):                  
#this is a weighted Binary Cross Entropy
    assert sigmoid_x.size() == targets.size()           
#make sure input and label have same shape
    count_p = (targets == 1.0).sum() + 1                 
    count_n = (targets == 0.0).sum() + 1                
#count 1(positive) and 0(negative), add 1 to avoid 0
    loss = -((targets * sigmoid_x.log()) * (1 / count_p)) - (((1 - targets) * (1 - sigmoid_x).log()) * (1 / count_n))
#divide the positive part and negative part with their count respectively
    return loss.mean()

then try it with a random input and all 0 label
if __name__ == '__main__':
    a = np.random.uniform(low = 0.0, high = 1.0, size = (3, 3))
    a = torch.from_numpy(a).float().to('cuda:0')
    a.requires_grad = True
    b = np.zeros((3, 3))
    b = torch.from_numpy(b).float().to('cuda:0')
    b.requires_grad = False
    loss = weighted_BCE(a, b)
    loss2 = torch.mean(-((b * a.log()) * (1 / 1)) - (((1 - b) * (1 - a).log()) * (1 / 10)))
# calculate the loss in the same way without definition
    print(loss.item())
    print(loss2.item())
    print((loss - loss2).item())

# here is the result
0.0
0.15303052961826324
-0.15303052961826324 #this is the supposed to be zero

It may be a silly question, but I can’t figure out the reason why the loss func returns 0.0 in a definition and different result when calculate it in the main func.
My environment is
Os : Ubuntu 18.04.2 LTS
GPU: GTX 1080ti
pytorch: 1.2.0