Personally, I use the WeightedRandomSampler class (https://pytorch.org/docs/stable/data.html#torch.utils.data.WeightedRandomSampler 62) and give each sample a weight of inverse the frequency of each class, thus resulting in an equal representation of the classes in the mini-batch (assuming a big enough batch size I guess)