How would these pairs be generated? Would samples be redrawn from the dataset or would all combinations be used?
I think the proper way would be to write a custom sampler and implement your sampling logic there.
The sampler passes its indices to Dataset.__getitem__ to load each sample and could pass indices for each pair to it.