If you want to select specific samples, I think you should implement a custom sampler.
Samplers are used to create the indices, which are passed to the Dataset.__getitem__ and can thus implement e.g. shuffling or weighted sampling.
Once the samples are loaded they will be passed to the collate_fn, which will then create a batch (by default it will just torch.stack the samples).
Would that make sense or am I still misunderstanding the use case?