I haven’t found an opitmal solution, for the meantime I settled for a smaller dataset.
At first I tried to use John second option (the get item load each sample separately), but it was really slow on my computer (about 28 sec to load each sample).
Than I tried to get creative and divide each epoch into 8 parts (of 1000 samples each), and after each part I deleted the dataloader object and created a new dataloader object with a different dataset, like that:
for j in np.arange(25):
    ########Starting train epoch######### 
  running_loss = 0
  train_loss_vector = []

  for k in range(8):    
      DB_T = My_Dataloader_Train(k)
      train_loader_train = data.DataLoader(dataset = DB_T, batch_size = 16, num_workers=0)
      
      net.train() # setting the model for training mode
      for batch_inx, (specs,masks) in enumerate(train_loader_train):

          specs = specs.to(device).float()
          .
          .

class My_Dataloader_Train(Dataset):
  def __init__(self,i):
    super().__init__()
    self.file_path = '/content/drive/My Drive/Project Dataset/trainFile' +str(i)+ '.h5'
    self.input = h5py.File(self.file_path, 'r')['Input'][:]
    self.target = h5py.File(self.file_path, 'r')['Target'][:]    
    self.len = 1000

It looked like it was working but after (I think) 10 epoches colab crashed with an ‘Unable to open file’ error (which is weird because it already succeeded opening the same file an epoch ago.
Anyway, I’m still wating for a better solution.