Shuffle does nothing but shuffling the indices which are gonna be called. Seems that you have troubles reading non-sequential data. Why don’t you try to change your data format?
Besides, I don’t usually use h5py but it seems you are reading  the whole file in getitem, if it’s very big that may be problematic.