I’d format my dataset similar to others that have similar objective like the librispeech: librispeech  |  TensorFlow Datasets 3
That will help you train a model later as there are many examples already based on the the librispeech dataset