Thank you for your kind reply.
torch.unique is a good idea to use instead of class labels. I never thought about that.
" if the current batch only contains a single sample of a class, your  nn.BatchNorm1d  layer(s) will throw an exception, since they cannot calculate the batch stats for a single sample (and 1 sequence length)." exactly. but the model will be trained on the balanced dataset.
I have a fundamental question regarding the loss backward. The final loss is the summations of all losses. I think scalar loss value will update the parameters of individual models that are connected by the closed graph of specific input labels.  Is the assumption correct?