Ten epochs seems like a rather high number for training T5 to me. How are the training and validation losses evolving throughout the training?
In case you don’t already, you could implement some early stopping mechanism in the training so that the training stops if the model starts getting worse, and just load the nest model at the end: Trainer
Hope that heps.
Cheers
Heiko