Hi All,
I was just wondering if there’s a possible extension to hooks in order to get the the hessian of the loss with respect to the output a layer. For example, using register_full_backward_hook returns the variable grad_output which represents the derivative of the loss with respect to the module’s output which I’ll denote dL_dsi (for the output, s, of the i-th layer of a network). And, is the same shape as the output of that layer).
I was wondering if it’s at all possible (or indeed even feasible) to take that grad_output and find the derivative of that w.r.t to another grad_output variable. So, in fact, calculate d2L_dsi_dsj where si and sj represent the output of the i-th and j-th layer respectively.
Any suggestions or help would be greatly appreciated!
Thank you!