Hi Anyere!



 Bendrien:

I am recomputing the weights for each minibatch. Does the shown computation of the weights represent the occurences of my labels properly for the CrossEntropyLoss?


With one correction, your weight computation looks correct.
Note, that in the case that hist.min() == 0 (perfectly possible), all
of your weights will be zero.  I would eliminate m = hist.min(), and
go with 1.0 / v (in place of m  / v).
A couple of comments:
I don’t think that the overall normalization of the weights matters.  I
think that if you use reduction = 'mean' in your CrossEntropyLoss
(the default), the overall scale factor of your weights will drop out.  (I.e.,
something like weight = 123.4 * weight won’t have any effect.)
I would probably just calculate the weights for the entire training set,
and not bother doing it on a per-batch basis.  (I think both approaches
are reasonable.)  As you take multiple steps in your gradient-descent
optimization algorithm, you are, in a sense, averaging over multiple
batches, so there is nothing magic about reweighting individual batches.
Last, using weight-of-class = 1 / number-of-samples-in-class
is perfectly reasonable, and I think somewhat standard.  But other
choices are okay.  You might experiment with only partially reweighting
your classes.
Good luck!
K. Frank