Hi, I’m trying to get a hessian vector product of a network (for TRPO). But the following codes don’t work as expected. Do I miss something important? Does anybody know how to solve it? Thank you in advance.
Environment

Mac and Ubuntu
Python 3.6
PyTorch 0.4.0

Conv case
conv = nn.Conv2d(3, 64, 3)
input = torch.randn(1, 3, 32, 32)
out = conv(input).sum()
grads = autograd.grad([out], conv.parameters(), create_graph=True)
flatten = torch.cat([g.reshape(-1) for g in grads if g is not None])
x = torch.randn_like(flatten)
print(flatten.shape) ## torch.Size([1792])
hvps = autograd.grad([flatten @ x], conv.parameters(), allow_unused=True)
print(hvps[1]) ## None
flatten2 = torch.cat([g.reshape(-1) for g in hvps if g is not None])
print(flatten2.shape) ## torch.Size([1728])

In this case, the gradients of conv.bias are None.
Linear case
linear = nn.Linear(10, 20)
input = torch.randn(1, 10)
out = linear(input).sum()
grads = autograd.grad([out], linear.parameters(), create_graph=True)
flatten = torch.cat([g.reshape(-1) for g in grads if g is not None])
x = torch.randn_like(flatten)
print(flatten.shape)
hvps = autograd.grad([flatten @ x], linear.parameters(), allow_unused=True)

Here, I got the following message.
Traceback (most recent call last):
  File "fvp.py", line 24, in <module>
    hvps = autograd.grad([flatten @ x], linear.parameters(), allow_unused=True)
  File "/Users/foo/.miniconda/lib/python3.6/site-packages/torch/autograd/__init__.py", line 144, in grad
    inputs, allow_unused)
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn