Hi,
As soon as you use detach, you don’t get “real” gradients and the whole thing becomes quite hard to reason about.
The main reason for you to get the same value, is that in the lower part of your function, the gradient does not depend on the value. Try changing your loss to a squared norm, and you will see a different behavior.
You can check this work in progress of the revamp of the autograd doc that tries to define detach in a more mathematical way: here 11.