craigyang:

Thus I wonder if I can save the not only the gradient value, but also its associated computation graph (both generated from grad_ft = torch.autograd.grad(loss_quad, model.parameters(), create_graph=True)) to a file or buffer, and reload it in a later iteration of the for loop.


The short answer is: I think so! Using saved tensors default hooks.
The docs are still under review (#62362 5 and #62361 2) but the functionality is already merged to master as of today!
In particular, the first PR describes exactly your use case where you want to save a computation graph to the disk and retrieve it later when needed.
I think in your case you would want to do something like:
# compute the flattened gradient with create_graph=True and store the graph on disk
torch.autograd.graph.set_saved_tensors_default_hooks(pack_hook, unpack_hook)  
model = quadratic_fun()
loss_quad = model.forward()
grad_ft = torch.autograd.grad(loss_quad, model.parameters(), create_graph=True)
flat_grad = torch.cat([g.contiguous().view(-1) for g in grad_ft])

# generate the constant matrix V, and compute the matrix-gradient product
torch.manual_seed(0)
V = torch.randn((10, 3))
h = torch.matmul(flat_grad, V)
torch.autograd.graph.reset_saved_tensors_default_hooks()

# compute the matrix-Jacobian product by iterating over the columns of the constant matrix
for i in range(3):
    hvp = torch.autograd.grad(h[i], model.parameters(), retain_graph=True)
    hvp_flat = torch.cat([g.contiguous().view(-1) for g in hvp])
    print(hvp_flat)


where pack_hook and unpack_hook are defined here 3.
You can control exactly which part of the graph should be saved to disk by adapting the position of the calls to set_saved_tensors_default_hooks and reset_saved_tensors_default_hooks.
Alternatively, you use the context manager torch.autograd.graph.save_on_cpu, cf #62410 2.