hey @treeofknowledge this plot suggests your learning rate is too high after 25k training steps so you could try using a learning rate scheduler like the default provided in transformers.Trainer (see here 13)