Hi all,
Could you please let me know if there is anyway to calculate the Gauss-Hessian matrix ?
Gaussian Newton is a quasi-Newton method which is defined here 36. It does not calculate direct the Hessian but approximate the Hessian buy broastcast product of two gradients as following function.
Pytorch does calculate gradient, but it seems like a sum of gradients over all input vectors. Can I have a matrix of gradients wrt one variable that NOT SUM, so I can manipulate the gradients later ?