Right now my model uses a single dataset for training and validation.
This is problematic for the model to learn representations of each label because of the nature of the data for some reasons:

Multi labels are somewhat rare ~ 5 %
Sentences with different labels are mostly very different

I think the performance of the model would be a lot better when the model uses label specific training and validation data to include more difficult cases for each label.
Thus, I am asking whether this is possible with transformers library or it is not benefitial at all towards performance.