I think I know what’s happening here. It takes a little time for the memory to clear so that it can be reused.
This is what worked!
        try:
            print("\nTrying to load data onto memory")
            data_loader = DataLoader1(trg_file_path, params)
        except RuntimeError as e:
            print ("Not enough memory to load all the data to GPU\nTrying generator approach")
            _initialized = 0
            while _initialized != 1:
                _initialized -= 1
                try:
                    data_loader = DataLoader2(trg_file_path, params)
                    _initialized = 1
                except RuntimeError as e:
                    if _initialized > -5:
                        print ("Failed to initialize, attempt {}. Let's try again".format(abs(_initialized)))
                    else:
                        raise RuntimeError(e)

Output:
Trying to load data onto memory
Preparing data from file = trg_data.txt
Not enough memory to load all the data to GPU
Trying generator approach
Preparing data from file = trg_data.txt
THCudaCheck FAIL file=/pytorch/aten/src/THC/generic/THCTensorMath.cu line=35 error=2 : out of memory
Failed to initialize, attempt 1. Let's try again
Preparing data from file =trg_data.txt

Success!!

As you can see, attempt 1 failed but attempt 2 suceeded. So now is there a way I can wait for memory to sync rather than running a while loop?