I encountered some unexpected behavior when using torch.autograd.grad https://pytorch.org/docs/stable/_modules/torch/autograd.html#grad 12
Here is a toy example:
x = torch.tensor([5, 3], dtype=torch.float, requires_grad=True)
y = x
print(torch.autograd.grad(outputs=y, inputs=x, grad_outputs=torch.ones_like(y), create_graph=True))

The output is correct:
(tensor([ 1., 1.]),)
However, if I tried to handle a sequence of tensors at a time, the result is unexpected:
torch.autograd.grad(outputs=[y,y], inputs=[x,x], 
                          grad_outputs=[torch.ones_like(y), torch.ones_like(y)], 
                          create_graph=True)

I would expect the result to be:
(tensor([ 1., 1.]), tensor([ 1., 1.]))
But, the actual result is:
(tensor([ 2., 2.]), tensor([ 2., 2.]))
I would assume the calculation of gradients is independent when the inputs/outputs of autograd.grad function are sequence of Tensor, but it appears not.
Here is a slightly more complicated example:

image.png1358×396 52.8 KB

I am not sure if this is the intended behavior. Any suggestion is appreciated.