I think a custom Dataset might be the easiest way.
Alternatively, you could write a custom transformation using e.g. RandomApply 3 as the base code and add the condition on the target.
However, this would also need the target as an input, which would most likely not work as a drop-in replacement for the vanilla transformations in torchvision models.