Solved by albanD in post #2 
              
              
                Hi, 
Optimizers don’t do that in general no. They will ignore Tensors whose .grad field is None and use the gradient that is there if there is one.