The reason is that I’ve tried to use data loader, but does not work for my specific case. I’m doing a training where I cannot use stochastic gradient descent, and I must use batch gradient descent which means every iteration I have to process the whole dataset to get the value output, and back propagate through the value output with extra gradient. I’ve tried to divide the data into different batches but it didn’t work for my case.
I’ve got error like “torch: not enough memory: you  tried to allocate 2GB.” I have 64 GB of memory, why I still do not have 2GB? That’s not true since I’m training entirely on cpu and not using GPU memory.