This use case might be possible to implement using a custom sampler.
You would need to create the mapping between each sample and the class as well as the current cluster.
Using this mapping, you could then implement a custom sampler by deriving from the base class here 12.
The sampler will return the indices for the current sample or batch.
It might be easier to derive from BatchedSampler 6 and pass a batch of indices into the Dataset instead of single indices.