Solved by ptrblck in post #2 
              
              
                You could either define a custom transformation, which accepts the image and target or use e.g. the segmentation transformations, which also accept these two arguments.