same question in

  
    
    
    Parameters with requires_grad = False are updated during training autograd
  
  
    Hello. I’am trying to freeze front layers during training. 
Before starting optimization, the optimizer is constructed by 
optimizer = torch.optim.SGD(net.parameters(), lr, ...)

Then, during training, i changed the front layers’ requires_grad=False. 
Specifically, 
for epoch in range(total_epoch):
  if epoch == freeze_epoch:
    net.conv1.weight.requires_grad = False
  train_one_epoch() #update network in an epoch


However, I found that the weights of the front layers are still updated. 
I als…
  


momentum, weight decay affect updating params with requires_grad=False