My main question is how to calculate the second order derivatives of a loss function. But I started with a toy example as follows:
import torch
x = torch.tensor(1., requires_grad = True)
y = 2*x**3 + 5*x**2 + 8
y.backward(retain_graph=True, create_graph=True)
x.grad
y.backward()
x.grad

My thought is by call “backward()” twice would give me the second order derivatives, but the answer is 32 which is not correct. It would be really appreciate if someone can explain where is wrong.
And second my goal is to calculate the second order derivatives of the following loss function. I have this following defined loss function, and I tried to use newton’s method to do optimization. Since newton’s method requires the first derivative and second derivative at the each iteration, so I tried to write some code as follows:
loss.backward(retain_graph=True, create_graph=True)
first_derivative = w.grad
loss.backward()
second_derivative = w.grad

I guess what I’m doing here is wrong given the above result of the toy example. How can I get the second order derivatives?

Screen Shot 2020-03-02 at 9.10.41 PM1014×132 6.75 KB