Hello Neta!



 netaglazer:

hi, i have a multi label problem


One good way to think about a multi-label classification problem
is to understand it as a set of binary classification problems (in
your case, five binary classification problems) that all use the same
input run through the same network.  That is, for a given input, each
of your five labels is either absent or present, so your network is
making five yes-or-no (binary) predictions.


but i minimize the loss even when the model was right in only one class


Understanding your problem as a set of binary classifications, we
see that the appropriate loss function is BCEWithLogitsLoss (or,
less optimally, BCELoss).  These loss functions have support for
multi-label problems built in.


i mean, if the model was right in one class:
label = [1,1,0,0,1]
predication = [1,0,0,0,0]


In this case I would say that three of your five predictions (one
“1” and two "0"s) were  correct, and two (two "0’s that should have
been "1"s) were wrong.


this is a success (loss=0)


This is a partial success (three out of five) but not perfect.  So your
loss should not be 0 (assuming that 0 is the minimum value of your
loss function).
BCEWithLogitsLoss will, indeed, partially penalize this kind of
three-out-of-five prediction.


im not sure how to calculate the accuracy in that case
when i should count a predication as +1 for acuuracy?


(Just to be clear, loss and accuracy are different things.)
I would count each correct prediction towards your overall
accuracy.  If it turned out that all of your samples produced
three-out-of-five-correct predictions (an unlikely artificial
assumption), then your accuracy would be 60%.
Just a quick note about the specifics of BCEWithLogitsLoss:
Your labels (targets) are appropriately 0-or-1 binary class
labels.  (These can be understood as probabilities – 0% chance
of the label being present vs. 100% chance.  Furthermore,
BCEWithLogitsLoss will actually accept probabilities between
0 and 1 for its targets – but you don’t have to use it this way.)
Your predictions, however, should not be 0-or-1 class labels
(nor probabilities), but rather logits – that is “raw scores” that
range from -inf to +inf.  You would normally get these from
the last linear layer of your network with (in your case) five
outputs.
(You could use class labels / probabilities with BCELoss, but
using logits with BCEWithLogitsLoss is better.)
Best.
K. Frank