Hi @Herb, after this procedure how to you access the actual hessian vector products?
I tried the following, which, in approach, is similar to your way:
======
from torch import autograd
linear = nn.Linear(10, 20)
x = torch.randn(1, 10)
out = linear(x).sum()
grads = autograd.grad(out, linear.parameters(), create_graph=True)
flatten = torch.cat([g.reshape(-1) for g in grads if g is not None])
x = torch.randn_like(flatten)
print(flatten.shape, flatten.requires_grad)
flatten.requires_grad=True
p=(flatten @ x)
linear.zero_grad()
p.backward(create_graph=True)
for param in linear.parameters():
print(param.grad)
======
but this prints : (None, None)
Can you take a look?
Thanks,