I am planning to work on this project.
So, here is my first question
As the current dataset size is too small for it to effectively train BERT and planning on building my own dataset, by scrapping website that host content in Hindi.
The first thing that comes to mind is scrapping news article, which perpetual source of data, we can also tap into some social networks which have more content in Hindi, such as ShareChat. Hindi/Hindustani Literature websites
Does this approach sounds reasonable, or is too weak because common crawl could have already covered such websites?