Ilya_Kotlov:

I think that if I put requires_grad=False as you suggested actually loss2.backward(1) will backpropogated 1 backwards as the requires_grad=False, and then in the x’.backward(1.0), it will be called with 1.0 and not with the gradients of the second path.


No, that shouldn’t be the case and you could verify it by either calling backward() on the first output and compare the gradients in lin or by checking the intermediate gradient on the first output manually:
lin.zero_grad()
out = lin(x)
out.retain_grad()
out_ = F.linear(out, lin.weight.detach())
out_.backward()
print(lin.weight.grad)
print(out.grad)