Hi Rishav!



 Rishav_Sen:

One of my classes has 20,000+ entries, while the next one is 3,000 entries, and everything else is below 3,000 entries.


It does sound like your training data is sufficiently unbalanced that
some weighting scheme is likely to beneficial and perhaps even
necessary.
There are two common approaches (which can even be combined
together):
You can sample your less-common classes more heavily when you
construct your training batches.  A helper tool for doing this is pytorch’s
WeightedRandomSampler.
You can also weight the less-common classes more heavily in your
loss function.  If you are using CrossEntropyLoss, you would do
this using its weight constructor argument.
I don’t have any evidence for this, but my belief is that the first approach
will be preferable unless your less-common classes are so rare that you
find yourself with many duplicates of the same sample in any given
batch you construct with weighted sampling.
Best.
K. Frank