There are many examples showing how to train T5 in PyTorch, e.g.: https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb 75 , but none so far for Tensorflow. Many people have been asking on transformers: https://github.com/huggingface/transformers/issues/3626 8 .
Does anybody have a good notebook showing how to train T5 in Tensorflow?
Otherwise, I will try to translate @valhalla 's great notebook: https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb 75 to Tensorflow.