Solved by LeeDoYup in post #5 
              
              
                First of all, i am using the momentum in optimizer. 
So i understand, the optimizer could update the parameters after i changed the requires_grad=False. 
When i check the gradient is “after” calling optimizer.step(). 
As you mentioned, if the optimizer with momentum modify the gradients of paramete…