I used that tool to define the hessian of an MLP by hand, and the speedup was orders of magnitude faster compared to the functional autograd hessian… I can share code if you like.
Specifically, functional.hessian took 15.926970720291138 seconds, whereas my hand-derived version took 0.2118682861328125 seconds. Both returned the same results.
Wow, I wondered if maybe the batch iteration was causing the major slowdown, but when I compared the functional hessian on just a single sample vs. my hand-derived hessian on a batch of 64 samples, my version (despite the 64x disadvantage) still ran more than twice as fast.