Solved by smth in post #2 
              
              
                I would therefore like to calculate the hessian for each convolution kernel, which is significantly less complex. 

While this is mathematically less complex, the way that convolution layers are implemented in every framework is that the compute the full forward (or backward-grad / backward-weight)…