Solved by colesbury in post #2 
              
              
                The transfer learning tutorial has a relevant section: 
http://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#convnet-as-fixed-feature-extractor 
To summarize: 


Set requires_grad=False for all parameters you do not wish to optimizer. This avoids computing gradients for them: 
for p…