Do you mean “multi-label classification”, where each sample might belong to zero, one, or multiple classes or “multi-class classification”, where each sample belongs to a specific class only?
We have a few code snippets in this forum showing how to use a weighted loss or weighted sampling to counter the effect of an imbalanced dataset.