Hi,
I have not tried 2 different models, but in the similar context, I tried to normalize my images twice sequentially and then unnormalize them exactly in the reverse order but it did not give me the result I was looking for, So I decided to use normalization->unnormalization twice.
But I think you do not need to convert back tensors to PIL images then do some preprocessing again and convert them to tensors. You can do those process without converting to PIL even if have not a class like Resize for tensors, because it would be easy to implement.