Hello Shamoon!



 shamoons:

I don’t really have all of the samples loaded at once - only batch per batch.


Pre-process your training set.  Count the number of positive vs.
negative samples in your training set – or if this is too bulky or
expensive, in a representative sample of your training set – and
use these counts to calculate a single value of pos_weight that
you use for all of your batches.


And those batches are randomly loaded, so isn’t it the same to have my pos_weight be per batch and per dataset?


Well, for large enough batches, this will be approximately true.
But, as you’ve discovered, with your batch size this can fail
because a batch can can contain no positive sample, so you
get a zero-divide when you try to calculate a per-batch
pos_weight for such a batch.
Best.
K. Frank