See how transformers.modeling_bert.BertForSequenceClassification is implemented - there is a CrossEntropyLoss built into this class internally. You can create custom class (PyTorch module) similar to BertForSequenceClassification and apply multi-label loss of your choice there. You can also look-up my blogpost https://zablo.net/blog/post/custom-classifier-on-bert-model-guide-polemo2-sentiment-analysis/ 202 where I show how to build totally custom model on top of the pre-trained transformers models.