For future reference, theres a pytorch package called Backpack which could be useful: GitHub - f-dangel/backpack: BackPACK - a backpropagation package built on top of PyTorch which efficiently computes quantities other than the gradient. 9