You need to create a model which inherits from PreTrainedModel (you’ve done it). In the forward, you have to define how you want the loss to be computed, then return loss along with logits. If you do this way, then you can use the Trainer directly.