Hi, I’ve done second order derivates on RNNs, is this what you want?
x_dim=10
y_dim=5
model=torch.nn.RNN(x_dim, y_dim, bias=False)
y,_=model(torch.randn(1, 1, x_dim))
loss=y.sum()
grad = torch.autograd.grad(loss, model.parameters(), create_graph=True)
grad_2nd = torch.autograd.grad((grad[0]**2).sum()+(grad[1]**2).sum(), model.parameters(), create_graph=True)