It makes sense that it takes as much time as the full hessian no? You evaluate each row of the hessian and only take the one element you want from it.
Note that you don’t do it explicitly, but that’s what torch.autograd.grad(first_derivative[i][j],  inp) is doing as it would be the same as
mask = torch.zeros_like(first_derivative)
mask[i][j] = 1
torch.autograd.grad(first_derivative, inp, grad_output=mask)