+1. Same issue here. Training performance varies for each batch, some batches are loaded very quickly, some take an unacceptable amount of time.
Have 3000 .npy files with multiple samples in each. Unable to find a scalable solution for building a map-style dataset when samples are distributed across multiple files on disk.