I think issue could best be described by giving a simple example. In the following simple script, I’m trying to take the Hessian-vector product where the Hessian is of f_of_theta taken w.r.t. theta and the vector is simply vector.

import torch
from torch.autograd import Variable, grad


theta = Variable(torch.randn(2,2), requires_grad=True)
f_of_theta = torch.sum(theta ** 2 + theta)
vector = Variable(torch.randn(2,2))


gradient = grad(f_of_theta, theta)[0]
gradient_vector_product = torch.sum(gradient * vector)
gradient_vector_product.requires_grad = True
hessian_vector_product = grad(gradient_vector_product, theta)[0]

gradient is being calculated correctly but when the script tries to calculated hessian_vector_product, I get the following error:

terminate called after throwing an instance of 'std::runtime_error’
what():  differentiated input is unreachable
Aborted

So, simply put, my question is how exactly should I do what I’m trying to do? Any help with this would be greatly appreciated.
Edit: Note that I’m using the pytorch version built from the latest commit on master (ff0ff33).