My batch size is 1, num_workers is 0, i think the problem is in how pytorch implements shuffling:
As i understood, it takes a list of indexes from 0 to len(dataset), then randomly permutes them, turns into a list, and then iterates over it to sample from the dataset
I think the fact that len() of my dataset is 2,548,407,963 is creating some problems, maybe this indexes list occupies all ram available?
EDIT:
And since my dataset is normalized to float32 (-1 to 1), it doesnt really matter if you load a list of int32 with length of len(dataset) for random sampling, or an entire dataset of same length, memory usage is the same
And if you use int 64, the list for shuffling itself would take up more space than dataset itself