So I have a network, lets call that model1. This is the model I want to train.
Similarity to Style Transfer, I want to use a different network (let’s call it model2, in style transfer model2 = VGG) as a feature extractor for the loss function.
So I put model2 in eval mode while training model1. However, even though I put model2’s parameters requires_grad= False, since it’s in eval mode i get the error “cudnn RNN backward can only be called in training mode”.
Is there a way to train model1 while model2 is in eval mode as a feature extractor?
Thanks