Hi @wonchulSon ,
Please find two threads about it here:

  
    
    
    How the pytorch freeze network in some layers, only the rest of the training? 
  
  
    Setting .requires_grad = False should work for convolution and FC layers. But how about networks that have instanceNormalization? Is setting .requires_grad = False enough for normalization layers too?
  



  
    
    
    Correct way to freeze layers 
  
  
    I would like to do it the following way - 
# we want to freeze the fc2 layer this time: only train fc1 and fc3
net.fc2.weight.requires_grad = False
net.fc2.bias.requires_grad = False

# passing only those parameters that explicitly requires grad
optimizer = optim.Adam(filter(lambda p: p.requires_grad, net.parameters()), lr=0.1)

# then do the normal execution of loss calculation and backward propagation

#  unfreezing the fc2 layer for extra tuning if needed
net.fc2.weight.requires_grad = True
n…
  


The second one is doing exactly what @DoubtWang is suggesting above. 
Let us know if you need more details about it!