Hi MngFrc!



 MngFrc:

The thing that was not allowing me to interprete my problem as a classification one was that my threshold is not 0.5, but definitely smaller (let’s suppose 0.1)


Where does this threshold come from?  Is it somehow given as
part of the problem, or is it something you choose?  In the typical
case, you won’t be using a threshold while training your model.


Can I ask you how, since my classes are unbalanced, can I train my NN?


The term unbalanced generally means that you have many
more examples of one class than another.  Two approaches
are to sample the less common classes more frequently when
you train, or to weight the less common classes more heavily
in your loss function.  (See, for example, the weight argument
passed to the constructor of BCELoss.)
But I’m not sure that this is what you mean by “unbalanced.”


When I have a label equal to 0.9 for example - or any value greater than the threshold - should I round it to 1? Or should I differently weight somehow the 2 terms belonging to the BCELoss?


You talk about having a label of “0.9”.  Therefore I understand
that your samples are not labeled with integer class labels (for
which 0 would mean that the sample is in class-“0” and 1
would mean that the class is in class-“1”), but rather, that your
labels are continuous values between 0 and 1 and represent
the probability that your sample is in class-“1”.
This is fine.  You want to train your model to predict the values
of the probabilities given by your labels.  Cross-entropy is a
measure of how dissimilar two probability distributions are, so
BCELoss is fully appropriate for this use case.


The fact is that if I have a label equal to 0.5, the BCE Loss will not decrease to zero by its definition


This is true, but it’s not a problem.  The pytorch optimizers don’t
care whether your loss is zero – they just drive the loss (using
some flavor of gradient descent) to lower values (and if you
run them long enough, perhaps to its minimum value).
If your labelled probability is 0.5, then BCELoss takes on its
minimum value when your predicted probability is also 0.5.


But also, I think that I should differentiate a label equal to 0.5 with respect to one equal to 0.9


You should.  And BCELoss does this.  If your label is 0.5,
BCELoss will prefer a predicted probability of 0.5.  And if your
label is 0.9, BCELoss will prefer a predicted probability of 0.9.
In summary – if I understand what you are doing – you are
working with a binary classification problem, but with the
(perfectly reasonable) feature that your labels are continuous
probabilities between 0 and 1, rather than being integer class
labels equal to either 0 or 1.
BCELoss is just fine for this use case, and there’s no need to
use any kind of threshold in your training.
(But to repeat my earlier comment, you’ll be better off using the
mathematically-equivalent BCEWithLogitsLoss.)
Good luck.
K. Frank