I have been experiencing really slow read times (when I call data.next() in my training loop) when I use Google Cloud Compute. However my school’s HPC is reading the same file 20x faster and I couldn’t find the reason behind this difference.
Any help is appreciated!
def __getitem__(self, idx):
        if self.f_open == False:
            f = h5py.File("../filename.hdf5", 'r')["delta_HI"]
            self.f_open = True

        f[c['x'][0]:c['x'][1],
              c['y'][0]:c['y'][1],
              c['z'][0]:c['z'][1]]

EDIT: The code snippet above doesn’t work. Had to change it to:
        f = h5py.File("../filename.hdf5", 'r')["delta_HI"]

        f[c['x'][0]:c['x'][1],
              c['y'][0]:c['y'][1],
              c['z'][0]:c['z'][1]]