HDF5 usually does a decent job of caching and handling IO. But if each item you’re fetching and feeding to your NN is quite small, and the net itself is also on the small side, you can still wind up in a scenario where most of the work is IO and Python bookkeeping/data munging and your GPU doesn’t have much to do.
As per my comment about lazy init, you should be able to get multiple workers running with
def __init__(...):
  self.archive = None

def _get_archive(archive_file):
  if self.archive is None:
    self.archive = h5py.File(archive, 'r', libver='latest', swmr=True)
  return self.archive

def __getitem__(...)
  archive = self._get_archive()
  ... 

Just make sure you don’t call any method on your dataset between creation and passing to the multi-worker loader init… for len, you may need to load the dataset once, calc the len, cache it, let that instance of the HD5py file expire, and then continue…