Thanks. I was looking at a solution that uses ConcatDataset and followed by a some sampler such as SubsetRandomSampler . I have a feeling that this solution will be cleaner and faster.
@ptrblck - any suggestions? I was using your hack that I linked in my OP.