Add code to pretrain T5 model from scratch
# 🚀 Feature request

The T5 model can significantly improve NLP task accuracies. However, the existing pretrained models are all in English. I'd like to pretrain T5 model on different language datasets from scratch.

Can you add code on pretraining T5 model? Thanks.





@patrickvonplaten Can we pre-train T5 from scratch on any task? I want to use it for Question Answering.<|||||>Many notebooks for T5 are now added to the community notebooks :-) <|||||>@patrickvonplaten can you share the notebook which show T5 pre-training if it is available ?<|||||>@patrickvonplaten as of today, I didn't find any notebook that is related to T5 *pretraining* in the [community notebooks collection ](https://huggingface.co/transformers/master/community.html#community-notebooks). Could you elaborate more on where there is a codebase to do the pretraining? Thanks!<|||||>> @patrickvonplaten as of today, I didn't find any notebook that is related to T5 pretraining in the community notebooks collection . Could you elaborate more on where there is a codebase to do the pretraining? Thanks!

Yes I agree there is no guide for pretraining
