Okay, so in my example:
loss_func1 doesn’t work because d(torch.norm(x))/dx = -x/norm(x)
loss_func2 works because it doesn’t use directly x but x-y which is stored somewhere else than x.
loss_func3 doesn’t work, although d(torch.sum(p*q)/dp = q (which doesn’t use p, that’s weird)
loss_func4 works because d(torch.sum(p+q)/dp = ones_like(p)
loss_func5 doesn’t work because d(torch.log(p/q))/dp = 1/p
So, it makes sense for most of them, except for loss_func3, unless my calculation is wrong…