At a high level, how exactly does the implementation of higher order derivs work? My understanding is that you write down your layer operation in terms of simpler operations which are passed to autograd, which is a sort of black box that computes gradients. But then why do we need to explicitly write down formulas for second derivatives (e.g. https://github.com/pytorch/pytorch/blob/master/torch/nn/_functions/thnn/auto_double_backwards.py 143)? And how are the higher order derivatives computed?