Thanks Victor for the explanation. I imported tempfile as you suggested and have the following code that works:
import torch
import tempfile

class quadratic_fun(torch.nn.Module):
    def __init__(self):
        super(quadratic_fun, self).__init__()
        self.x = torch.nn.Parameter(torch.ones(5, requires_grad=True))
        self.y = torch.nn.Parameter(torch.ones(5, requires_grad=True))

    def forward(self):
        loss = torch.norm(self.x) ** 2 + torch.norm(self.y) ** 2
        return loss

class SelfClosingTempFile():
    def __init__(self):
        self.fp = tempfile.TemporaryFile()

    def __del__(self):
        self.fp.close()

def pack_hook(tensor):
    sctf = SelfClosingTempFile()
    torch.save(tensor, sctf.fp)
    return sctf

def unpack_hook(sctf):
    sctf.fp.seek(0)
    return torch.load(sctf.fp)

torch.autograd.graph.set_saved_tensors_default_hooks(pack_hook, unpack_hook)
model = quadratic_fun()
loss_quad = model.forward()
grad_ft = torch.autograd.grad(loss_quad, model.parameters(), create_graph=True)
flat_grad = torch.cat([g.contiguous().view(-1) for g in grad_ft])

torch.manual_seed(0)
V = torch.randn((10, 3))
h = torch.matmul(flat_grad, V)

torch.autograd.graph.reset_saved_tensors_default_hooks()

for i in range(3):
    hvp = torch.autograd.grad(h[i], model.parameters(), retain_graph=True)
    hvp_flat = torch.cat([g.contiguous().view(-1) for g in hvp])
    print(hvp_flat)

However, I want to set retain_graph to be False in the third from last line: I cannot retain the graph because of memory limits (and that’s exactly why I want to save the graph when I created it). If I remove retain_graph=True from the above code, I would still get
RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

which is the same as not using the hook functions. Is it because I put reset_saved_tensors_default_hooks to the wrong position? How should I do what I want with all the hook functions here?