Thanks for your answer. I modified the loss function to torch.norm() and observed different gradients at the input.
However, I have difficulty understanding the following part of your answer:
“The main reason for you to get the same value, is that in the lower part of your function, the gradient does not depend on the value.”
Do you mind sharing pointers to resources that explain this part in more details?
Thank you!