That is a beautifully crafted reply @Yaroslav_Bulatov. Many thanks for the informative graphic! And the illustration of how to use register_hook, that should come in useful in hte future.
It shows what really happens: PyTorch has summed up the partial derivatives over the different outputs. Which is why you’ve got 0.5 in the graphic, but only a 1D vector of derivatives in the backward hook.
I would still argue my original example of logits - norm is a sum however.  And if I were to look at the partial derivatives individually, that sum would match. The problem was that I was looking at the sum of the derivatives (summed over outputs), which is what PyTorch gives us.
Anyway, cheers!