If you see the increase in memory usage during the dummy DataLoader loop, the issue might be in the Dataset and in particular how you are loading/storing the data.
Could you post the code for your Dataset? The TensorImageList class doesn’t look like it’s corresponding to Dataset, as the __getitem__ and __len__ methods are missing.