You could write a custom sampler by deriving from the base class from here 100 and apply your custom sampling logic.
The sampler will yield the indices, which will be passed to the Dataset.__getitem__, so you could sort these indices to make sure that the right images are loaded.