@Nikronic I might have not explained myself well / used the wrong terminology. When I wrote feature extractor I meant using the VGG on the outputs of model 1. for example:
model1.train()
y = model1(x)
features = model2(y)
features_truth = model2(y_ground_truth)
loss_on_deep_features = criterion(features, features_truth)

I wondered if using model2 in train mode will hurt the performance (give me incorrect features of x and x_truth). When i tried putting model2 in eval mode i got the above error, even though I put requires_grad = False for model2.
Anyway thank you for your answer and time!