Thanks for the link. In my case, this happens before training, the model is not created yet! In the try block I’m trying to load all the training set onto memory which sometimes fails. Since that object is not created shouldn’t the DataLoader2 in exception be excecuted?
torch.cuda.empty_cache(), does not seem to help here. Moreover the documention also states

“the occupied GPU memory by tensors will not be freed so it can not increase the amount of GPU memory available for PyTorch”