A tool like this would be really useful:
http://www.matrixcalculus.org/matrixCalculus 5
Check out the “export to Python” option. It spits out numpy, but if we had the same sort of tool for tensor calculus with Pytorch, that would make higher order derivatives so much more efficient. That way, a function can be provided as input and a function can be provided as output. Unfortunately, the tool I linked doesn’t do tensor calculus, only matrix calculus.