Hello Hyo and RaLo!



 Hyo:

For every observation I have 4-5  categories and total number of categories are 100.
For 1 observation the target labels are [1,3,56,71]  I have converted it into one hot vector representation





 RaLo4:

Like a heavily imbalanced dataset for example.


Yes, from Hyo’s post, this should be understood as a imbalanced
dataset.  This can be addressed with BCEWithLogitsLoss's
pos_weight constructor argument.
This is not necessarily imbalanced in the sense of, say, class 7 vs.
class 23 (might be, might not be – from what Hyo has said, we don’t
know yet), but it is imbalanced in the sense of the presence, say, of
class 7 vs. the absence of class 7.
Let me give a few words of explanation:
This multi-label, 100-class classification problem should be
understood as 100 binary classification problems (run through the
same network “in parallel”).  For each of the classes, say class 7, and
each sample, you make the binary prediction as to whether that class
is present in that sample.
Your class-present / class-absent binary-choice imbalance is (averaged
over classes) something like 5% class-present vs. 95% class-absent.
This is imbalanced enough that your network is likely being trained
to predict any one specific class being present with low probability.
It sounds like this is what your are seeing.
(The “standard” approach for using pos_weight would be to calculate
for each class c the fraction of times, f_c, that class c is present
in your samples (regardless of which other classes are present or
absent), and the calculate the weight w_c = (1 - f_c) / f_c.  You
then pass the one-dimensional tensor [w_0, w_1, ..., w_99] into
BCEWithLogitsLoss's constructor as its pos_weight argument.)



 Hyo:

In the accuracy_score I need to round of the values of the output to 1 and 0 how do I take the threshold?


A second comment:
The most straightforward way to convert your network output to
0 vs. 1 predictions is to threshold the output logits against
0.0.  You are certainly allowed to convert the logits to probabilities,
and then threshold against 0.5 (or, equivalently, round), but doing
so is not necessary.  More detail is given in this post:

  
    
    
    BCEWithLogitsLoss and model accuracy calculation 
  
  
    Hi Mehdi! 

You should use 0.0 as your threshold for class “1”. 
Here are the details: 
A logit is a sort of score that runs from -infinity to +infinity. 
When you run it through the sigmoid function, you get a 
probability that runs from 0 to 1. (BCEWithLogitsLoss has, 
in effect, a sigmoid function inside of it.)  We interpret this 
probability as being the probability of class “1”.   So we (usually) 
convert such a probability to a yes-no prediction by saying if 
the probability of being clas…
  


Good luck.
K. Frank