You could keep lists of pairs of filenames and labels and prepare batches asynchronously in a background worker thread. That should be efficient even when training with millions of images. You might wanna take a look at the  torch.utils.data.dataset.Dataset and torch.utils.data.sampler.Sampler classes that you can use in conjunction with the torch.utils.data.DataLoader.