Solved by ptrblck in post #4 
              
              
                Thanks for the information! 
Since you are loading each .npy in the Dataset.__init__ method, ConcatDataset will append all already initialized datasets and will thus use a huge amount of memory. 
Would it be possible to create a loop over each '.npyfile and create a singletrainDatasetinside this loo…