So, my class 0 is 20% and class 1 is 80% of the entire data and I have stratified 5 fold train/val/test with division of 60/20/20.
However, my predicted labels for test set are stuck at 1.
Any thoughts how to fix this? I used the weighted loss suggested in
  
    
    
    Unclear about Weighted BCE Loss 
  
  
    If you would like to use the weight as a class weight, I assume you have a binary target, i.e. it contains only zeros and ones. 
In that case I would create a weight tensor and just multiply it with your unreduces loss. Here is a small example: 
weight = torch.tensor([0.1, 0.9])
weight_ = weight[y.data.view(-1).long()].view_as(y)
criterion = nn.BCELoss(reduce=False)
loss = criterion(output, y)
loss_class_weighted = loss * weight_
loss_class_weighted = loss_class_weighted.mean()

Let me know, if …
  


except I am not sure if I used it correctly.
X here is an intermediate representation of NxD where N is number of patches in my images and D is resnet18 last layer feature, 512 1D tensor. My batch size is 16.
I am also unsure if pred_labels = out.argmax(dim=1) is correct or not? Also, couldn’t directly use BCELoss since my out looks like something like:
*one instance from val phase:
x is:  tensor([[ 0.0154,  0.0957],
        [-0.0446,  0.0787],
        [-0.0272,  0.0265],
        [ 0.1656, -0.1316],
        [ 0.0931,  0.0134],
        [ 0.2454,  0.2079],
        [ 0.5408, -0.0578],
        [ 0.2008,  0.1331],
        [ 0.1812,  0.0924],
        [ 0.1018,  0.0468],
        [ 0.1389,  0.2432],
        [-0.3082, -0.2986],
        [ 0.1031,  0.0219],
        [ 0.0910, -0.1239],
        [ 0.4165,  0.0890],
        [ 0.2290, -0.0611]], device='cuda:0')
loss:  tensor([0.6538, 0.6334, 0.7203, 0.5555, 0.7338, 0.7120, 1.0366, 0.7276, 0.7385,
        0.7210, 0.7466, 0.6884, 0.7346, 0.8064, 0.8703, 0.5585],
       device='cuda:0')
weight_:  tensor([0.9000, 0.9000, 0.1000, 0.1000, 0.9000, 0.9000, 0.9000, 0.9000, 0.9000,
        0.9000, 0.1000, 0.9000, 0.9000, 0.9000, 0.9000, 0.1000])
val pred:  tensor([1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0], device='cuda:0')
val label:  tensor([1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0])

self.criterion = nn.CrossEntropyLoss(reduce=False)
#self.criterion = nn.BCELoss(reduce=False)
def forward(self, X, labels):

    stacked_X = torch.stack(X)
    out = self.transformer(stacked_X)
   
    with torch.autocast('cuda'):
        # https://discuss.pytorch.org/t/unclear-about-weighted-bce-loss/21486/2 
        labels = torch.tensor(labels)
        weight = torch.tensor([0.1, 0.9]) # how to exactly decide on this weights?
        weight_ = weight[labels.data.view(-1).long()].view_as(labels)
        loss = self.criterion(out, torch.tensor(labels).cuda())
        print('loss: ', loss)
        print('weight_: ', weight_)
        loss_class_weighted = loss * weight_.cuda()
        loss_class_weighted = loss_class_weighted.mean()
   
   
    #pred = out.data.max(1)[1]
    
    pred_labels = out.argmax(dim=1)
    
    return pred_labels, labels, loss_class_weighted

If I use BCELoss with reduce=False instead, I get this error:
  loss = self.criterion(out, torch.tensor(labels).cuda())
Traceback (most recent call last):
  File "main_classifier.py", line 250, in <module>
    pred,label,loss = trainer.train(sample_batched, model)
    pred, labels, loss = model.forward(feats, labels)
    return self.module(*inputs[0], **kwargs[0])
  File "/home/jalal/research/venv/dpcc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
    loss = self.criterion(out, torch.tensor(labels).cuda())
  File "/home/jalal/research/venv/dpcc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/jalal/research/venv/dpcc/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 603, in forward
    return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
  File "/home/jalal/research/venv/dpcc/lib/python3.8/site-packages/torch/nn/functional.py", line 2906, in binary_cross_entropy
    raise ValueError(
ValueError: Using a target size (torch.Size([16])) that is different to the input size (torch.Size([16, 2])) is deprecated. Please ensure they have the same size.