I wanted to check your num_train_epochs wasn’t overridden by other parameters like max_steps. The code looks correct and the logs do indicate you are going through the epochs. Double check the length of your dataset to make sure it’s not reduced to something small.