Solved by albanD in post #6 
              
              
                For " the gradient information  w.grad  is treated as a number if the computation graph and not as a variable that one can differentiate" you want to use w.grad.detach() for further computations to achieve that. 
.data will work but can be misleading as it does not allow the autograd to perform all …