I looked a bit into it and the problem is the following:
If one loss becomes nan or inf all the following displayed losses also become nan or inf since the shown loss is the average of all losses seen so far, see: transformers/trainer.py at 82b8d8c7b02562695f88be81cf0993972e324874 · huggingface/transformers · GitHub 16
However this doesn’t mean that the losses after nan is displayed are actually useless → the model can very well train. So it’s more of a display error than an actual error often times. All in all my best suggestion here is to just take a look at the validation loss and if it goes down smoothly continue training