Solved by albanD in post #4 
              
              
                Yes you can use .detach() to get a new Tensor that does not share the gradient history with the original Tensor. 
Or if you want to do an op that should not be tracked by autograd (like a copy) you can do: 
with torch.no_grad():
    params2.copy_(out)

In your case, I think you can generate the grad…