If your whole dataset fits into your ram, you could load it from one large file inside your dataset’s __init__. This would load all your data once and take a bit longer for the initialization, but would be the fastest way during training, since no additional data has to be loaded.
If that is not possible, you could increase your number of workers and load the images inside your __getitem__. If I remember correctly, HDF5 works as a generator, which means you could also use one large HDF5 file. The advantage is that you don’t need that much RAM, on the other side you would have to load your data during training, which may be a bottleneck (depending on the size of your images, the performance of your GPU, your CPU and your number of workers).