Since I want to backprop through the resulting loss, I (think I) need to set create_graph in the second call to grad(). Otherwise, I am getting the following:
# loss = get_loss_fast(f, x, v)
# loss.backward()

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

where f is an nn.Linear, and x and v are two random vectors. When I specify create_graph=True, there is no such error.
In any case, as far as I understand, the performance penalty for either doing a .detach() or doing a redundant gradient computation is negligible, so there is little point in benchmarking whether this .detach() is useful or not.
A working version of the code for posterity:
def get_loss_fast(f, x, v):
    x = x.detach().requires_grad_()
    v = v.detach().requires_grad_()

    vjp_val = autograd.grad(f(x), x, v, create_graph=True)[0]
    jvp_val = autograd.grad(vjp_val, v, v.detach(), create_graph=True)[0]

    return (vjp_val - jvp_val).norm(1)