If I understand your use case correctly, you would like to apply the same random transformations to the image and target tensors.
In this case you could use the functional API as described in this example 2.