Solved by albanD in post #2 
              
              
                Hi, 
I am afraid you won’t be able to do that. 
The “requires_grad” property is associated with the whole Tensor. And cannot be set for subsets. 
You have a few approaches here: 

Compute the gradient for the full layer and then zero-out the gradients before doing the optimizer update
If your optimi…