Thanks for the suggestion :-). I’d prefer not to define the derivatives myself though, I think they could be quite complex, which is exactly why I’d like to make use of the automatic differentiation of pytorch.
The first and main question is whether repeating torch.autograd.functional.jacobian like in my post theoretically gives me the correct 3rd order derivatives?
The 2nd question, the underflow problem, I could perhaps solve by giving lambda x: torch.exp(scalar_func(x)) as argument instead of scalar_func, since scalar_func returns the log of the partition function? Or does this not make sense, for numerical stability?