T5 pre training on different languages from scratch
Hi Team,

I was exploring the pre training procedure/document for T5 models(small,base, large) models on different languages from scratch. However did not come across anything which could help me. Please share if there is any resource for the same. If not, could you please consider to include it .

Thank you .
Hi @ashispapu, T5 pre-training is not yet available in transformers, I'm working on it but might take some time. Feel free to take a stab.<|||||>Hi @patil-suraj Thanks for the response. Let me try it.<|||||>@patil-suraj, thanks for working on the different T5 ropes. Can you point to the branch you 're working for pre-training? I'd be interested to contribute.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I'm interested in T5 pre-training from scratch. Any update on this project?