Thanks, tried that but didnt work.
    try:
        print ("\nTrying to load all data onto Memory")
        data_loader = DataLoader1(trg_file_path, params)
    except RuntimeError as e:
        if 'out of memory' not in str(e):
            raise RuntimeError(e)
        print ("Not enough memory to load all that data\n\nTrying generator approach")
        torch.cuda.synchronize()
        _initialized = 0
        _max_retries = 5
        while _initialized != 1:
            _initialized -= 1
            try:
                data_loader = DataLoader2(trg_file_path, params)
                print("Attempt {}/{}, initialized".format(abs(_initialized), _max_retries))
                _initialized = 1
            except RuntimeError as e:
                if _initialized > -1 * _max_retries:
                    print("Attempt {}/{}, failed to initialize, trying again".format(abs(_initialized), _max_retries))
                else:
                    raise RuntimeError(e)

stdout
Trying to load data onto memory
Preparing data from file = trg_data.txt
Not enough memory to load all that data

Trying generator approach
Preparing data from file = trg_data.txt
THCudaCheck FAIL file=/pytorch/aten/src/THC/generic/THCTensorMath.cu line=35 error=2 : out of memory
Attempt 1/5, failed to initialize, trying again
Preparing data from file = trg_data.txt
Attempt 2/5, initialized

Still takes 2 attempts 
EDIT:Nomenclature