The loss calculation regarding real_b is done on a target sampled from your DataLoader, while val_real_B is still attached to the computation graph of your vgg_MODEL.
I assume you want to train the model to predict val_same_B to be as close as possible to val_real_B, so we only need the gradients based on the computation graph from val_same_B.