The new variable will reference the old one, so that both will see the value changes and gradients:
lin = nn.Linear(1, 1)
w = lin.weight

lin(torch.randn(1, 1)).backward()

print(lin.weight.grad)
print(w.grad)
print(id(w)==id(lin.weight))