Great! Created some of my project ideas here.

PreTrain GPT2 from scratch in Bengali
PreTrain T5 from scratch in Bengali
PreTrain RoBERTa (MLM model) from scratch for Programming Languages

Not sure what is the condition of the T5 pre-training script. Would love to contribute and adapt the given MLM and CLM script to T5 if it’s not done yet.