@codeflux


In your case there are three variables x, y and v


x is the foremost input and y is the final output. v is an intermediate calc


In an ideal scenario, if you have a single value for x ( e.g. x=5 ) ,the values for v and y are fixed ( immutable ) and therefore the grads are fixed for a single loss backward propagation


grad is designed to start from the calculated loss(which is a scalar value) and then propagate backwards for each variable, which you have already seen. If the loss is the same, the grads will also be the same


If you want to alter the grads, you need to register hooks so that instead of the normal grads propagating backwards, you have your own modified grads propagating


https://pytorch.org/docs/stable/generated/torch.Tensor.register_hook.html
WITHOUT HOOK
x = torch.ones(2, 2, requires_grad=True)
gt = torch.ones_like(x) * 16 - 0.5  # "ground-truths"
# We will use MSELoss as an example.
loss_fn = nn.MSELoss()
# Do some computations.
v = x + 2
y = v ** 2

# Compute loss.
loss = loss_fn(y, gt)
d_loss_dx = grad(outputs=loss, inputs=x, retain_graph=True)
d_loss_dv = grad(outputs=loss, inputs=v, retain_graph=True)
d_loss_dy = grad(outputs=loss, inputs=y)

print(f'dloss/dx:\n {d_loss_dx}')
print(f'dloss/dv:\n {d_loss_dv}')
print(f'dloss/dy:\n {d_loss_dy}')

dloss/dx:
 (tensor([[-19.5000, -19.5000],
        [-19.5000, -19.5000]]),)
dloss/dv:
 (tensor([[-19.5000, -19.5000],
        [-19.5000, -19.5000]]),)
dloss/dy:
 (tensor([[-3.2500, -3.2500],
        [-3.2500, -3.2500]]),)

WITH HOOK
x = torch.ones(2, 2, requires_grad=True)
gt = torch.ones_like(x) * 16 - 0.5  # "ground-truths"
# We will use MSELoss as an example.
loss_fn = nn.MSELoss()
# Do some computations.
v = x + 2
y = v ** 2

h = v.register_hook(lambda grad: grad * 2)

# Compute loss.
loss = loss_fn(y, gt)
d_loss_dx = grad(outputs=loss, inputs=x, retain_graph=True)
d_loss_dv = grad(outputs=loss, inputs=v, retain_graph=True)
d_loss_dy = grad(outputs=loss, inputs=y)

print(f'dloss/dx:\n {d_loss_dx}')
print(f'dloss/dv:\n {d_loss_dv}')
print(f'dloss/dy:\n {d_loss_dy}')

h.remove()

dloss/dx:
 (tensor([[-39., -39.],
        [-39., -39.]]),)
dloss/dv:
 (tensor([[-19.5000, -19.5000],
        [-19.5000, -19.5000]]),)
dloss/dy:
 (tensor([[-3.2500, -3.2500],
        [-3.2500, -3.2500]]),)

There are different variations of applying hook, and you can choose based on what suits you