Hi Liran and Isaac!



 liran:

I tried to set weight of 500 for the positive labels (that`s also the minority class)


As an aside, You mention in a later post that your positive class is
about 3% of your data.  You might start out then by setting pos_weight
to about 30, so that (after weighting) your negative and positive classes
are about equally weighted.  This won’t necessarily be the best choice
for pos_weight, but it’s a good place to start, and you should certainly
compare it with other values you try (such as 500).


if I change it to:
criterion = nn.BCEWithLogitsLoss(pos_weight=None)
I get much smaller value on the loss function


This is not necessarily a problem.  With pos_weight = None, your
model could easily train to predict your negative class (almost) all
of the time, and almost always be right.  So you get a small loss.
When you use pos_weight to increase the weight of your
under-represented positive class, your model actually has to learn
to tell the two classes apart, so you are likely to get a larger, but
more meaningful loss.



 Isaac_Kargar:

Here you set the weights to 500 and you have one class only? Shouldn’t it be 2? Check this  please.


I think this doesn’t apply to Liran’s use case.  It sounds like he has a
single-label, binary classification problem.  So pos_weight should
have a size of 1 rather than 2.
(The terminology is a little bit confusing.  When we talk about a binary
problem, we do say that there are two classes.  But when we talk
about a multi-label, multi-class problem, we talk about nClass
classes.  But the multi-label problem should be understood as
nClass binary problems (that share the same network) – that is,
each of the nClass labels can be absent or present.  The example
you link to is a 64-class multi-label, multi-class problem.  It therefore
consists of 64 binary problems, each one having its own value of
pos_weight.)
Best.
K. Frank