Ok I figured it out. Not a problem with anything in the framework. I had a random transformation in the create transform. One output corresponds to the flipped image and the other to the original.
Can someone please delete this. Thank you.