Thanks for the information!
Since you are loading each .npy in the Dataset.__init__ method, ConcatDataset will append all already initialized datasets and will thus use a huge amount of memory.
Would it be possible to create a loop over each '.npyfile and create a singletrainDatasetinside this loop for the current.npy`?
This would hold a single dataset in memory at any time and should reduce the memory foot print significantly.