At least according to this, hdf5 format is not great as it accumulates over time. Is there a better format these days?
  
    
    
    Save torch tensors as hdf5 vision
  
  
    HDF5 is not a great format for appending information over-time… It will end up generating a very large binary file to handle new data. 
I’d recommend doing it for a fixed size. E.g. first create a dataset of a fixed size: 
N = 100 # find the length of my dataset
data = h5_file.create_dataset('data', shape=(N, 3, 224, 224), dtype=np.float32, fillvalue=0)

Then populate it 
for i in range(N):
    img = ... # load image
    data[i] = img

h5_file.close()

However, if you really really want to do re…