Hello Yong Kuk!



 ykukkim:

I am trying to utilise BCELoss with weights, but I am struggling to understand.
…
My datasets are imbalance, meaning that I do not have a constant length of the dataset as well as there are more 0’s than 1’s, approximately 100:1,


The most straightforward way to do this (and also better for numerical
reasons) is to adjust your network so that it outputs raw-score logits
for its predictions, rather than probabilities.  (For example, if the last
layer of your network is a Sigmoid – that converts a logit to a
probability – just get rid of the Sigmoid layer.)
Then use BCEWithLogitsLoss instead of BCELoss.  This is because
BCEWithLogitsLoss offers a pos_weight argument that it uses to
reweight positive samples in the loss function.  In your case you would
set pos_weight to something like 100.  (BCELoss does not have a
pos_weight argument – probably just an oversight, rather than for
any particular reason.)
For some further details, please take a look at this recent thread:

  
    
    
    Binary segmentation BCEWithLogitsLoss pos_weight? 
  
  
    I’m segmenting foreground vs background using unet and there are many more 0s than 1s due to this. It looks like    i should be using BCEWithLogitsLoss as my loss function? 
It looks like this function takes as an argument the proportion of class imbalance - “pos_weight (Tensor, optional) – a weight of positive examples. Must be a vector with length equal to the number of classes.” 
Does that mean i need to know across all my images how many pixels are 1 v 0  If so is there any best…
  


Good luck!
K. Frank