GPT2’s forward has a labels argument that you can use to automatically get the standard LM loss, but you don’t have to use this. You can take the model outputs and define any loss you’d like, whether using PyTorch or TF2. If you want to use Trainer, just define your own PT module that returns your custom loss as the first element from forward. See training and fine-tuning 135 and how to train a language model 80.