Hey @Modfiededition,
That’s a great question!
As a start, I think it always makes sense to see what datasets are already publicly available that you could use for your language. You could, e.g. see the Hugging Face Hub here and select the speech-processing tag: Hugging Face – The AI community building the future. 36 and then also your favorite language tag → then you can see which datasets are available through the Hub.
Apart from this you can also check out this github page that lists a lot of publicly available speech datasets: GitHub - jim-schwoebel/voice_datasets: 🔊 A comprehensive list of open-source datasets for voice and sound computing (95+ datasets). 62
If you want to extract audio/transcripts yourself, this is much more difficult and you also need to be careful about licensing. YouTube could be a good source, if the licenses allow it