Hi
My training data has the shape torch.Size([330000,7225]) and is currently saved and retrived by using torch.save() and torch.load() before being split into batches using TensorDataset and DataLoader . However when loading i get the error:
RuntimeError: [enforce fail at ..\c10\core\CPUAllocator.cpp:47] 
((ptrdiff_t)nbytes) >= 0. alloc_cpu() seems to have been called with negative number: 18446744066066682432

I suspect this is because the file is too large to be loaded all at once (it is 8.88 GB). Is there an alternative way in which I don’t have to load all of the training data first or would you recommend I save my data using an alternative approach?