Hello!
Great to see enthusiasm for Sinhala language. We have a RoBerta model for Sinhala on Hugging Face, which I trained on OSCAR dataset (which is a ~800MB dataset). I have a proposal to train a GPT2 for Sinhala with OSCAR and C4M dataset (which has ~3GB of Sinhala data). feel free to join : PreTrain GPT2 for Sinhala from scratch 2 I think this could be a very good downstream task.
Thanks!