Hello,
I have tried using the following custom loss class:
# class BCEWithLogitsLoss():
class CustomWBCE():
    def __init__(self, class_weights=None, **kwargs):
        self.class_weights = class_weights

    def __call__(self, output:Tensor, target:Tensor, **kwargs)->Rank0Tensor:
        output = torch.sigmoid(output)
        
        if output.min() <= 0 or output.max() >= 1:
#             print('Wrong value in output, will give loss nan')
            output = torch.clamp(torch.sigmoid(output),min=1e-8,max=1 - 1e-8)
        
        if self.class_weights is not None:
            assert len(self.class_weights) == 2

            loss = self.class_weights[1] * (target * torch.log(output)) + \
                   self.class_weights[0] * ((1 - target) * torch.log(1 - output))
        else:
            loss = target * torch.log(output) + (1 - target) * torch.log(1 - output)

        loss = torch.neg(torch.mean(loss))

        return loss

The reason I apply the sigmoid is because I am using this in a fast.ai Learner, which does not apply an activation function at the end of the network by default. Sometimes my model outputed a very small value (-136 for example) and torch.sigmoid’s result was 0., which led to a -inf in the torch.log; that’s why I added the torch.clamp.
I have tried it both with weights computed with the formula:
total = negative + positive
w0 = positive / total
w1 = negative / total

And with weights (0.5, 0.5) to test it out.
Both times I got very poor results:
AUC scores (this ranges from 0 to 1, 0.5 being random) after training with different weights:

Calculated weights (0.20, 0.80): 0.51 AUC
Equal weights (0.5, 0.5): 0.45 AUC (worse than random somehow)

Now, if I try to train the model in the same way, but using
loss_bce = fastai.layers.BCEWithLogitsFlat() (flattens the tensors before applying torch.nn.modules.loss.BCEWithLogitsFlat) it gets to 0.83 AUC.
The model is a DenseNet121 based binary classifier; I trained it on X-Rays from the CheXPert14 dataset.
I am on PyTorch version 1.0.1 and fast.ai version 1.0.52.
Do you have any idea what I’m doing wrong?