Thanks a lot!  I’ll read the in-progress doc around 20-30 times, maybe that’ll help hehe.  I’ve read the PyTorch papers, one is pointing towards the AD book “Evaluating Derivatives
Principles and Techniques of Algorithmic Differentiation, Second Edition” by Andreas Griewank and Andrea Walther. I’ll try to pick up the reverse mode AD from there also.
I want to work with higher derivatives of the models, so I really must be 100% sure about what grad is doing and how to build up on that.