I don’t think there’s anything that should stop your dataset from doing so? Just make sure the length is set appropriately (i.e. the maximum number of chunks it can return), and then have the relevant index point to a specific chunk. e.g. if you have 10 10-second clips and your timestep is 1s, you want to calculate length as 100 (and return that), then when asked for index 55 you’d want the 6th chunk of the 5th file (because 0 indexing). Everything should still work with batching and shuffled orders etc. as normal, then.
The fastest way would probably be to load all the audio clips into memory and split them first when initialising the dataset, that way it’s a normal array access, but that would be memory intensive.