Based on the description it seems that each batch would have specific requirements, since each forward pass would expect to use a special task.
If that’s the case, I think the cleanest way would be to create a custom sampler and precalculate the indices based on your task logic (or pass the precalculated indices to a SequentialSampler).