Do you want the full hessian matrix? If so, you will need to “reconstruct” it by doing multiple times the second backward pass with a grad_output that will be a one-hot matrix for each entry in the 2D Tensors one after the other.