Solved by crcrpar in post #2 
              
              
                I think nn.Parameter(w0.clone().detach()) will fix the problem. 
notebook: nn-parameter-changes-input-tensor.ipynb · GitHub 
Reference: See warning block of new_tensor torch.Tensor — PyTorch 1.8.0 documentation