I am trying to get the hessian after a model has completed estimating (to then use to calculate standard errors). Given a model object, my thought was that something like the following should work:
d1 = torch.autograd.grad(loss, model.parameters(), create_graph=True)[0]
d2 = torch.autograd.grad(loss, d1, create_graph=True)[0]

However, this gives the error “One of the differentiated Tensors appears to not have been used in the graph.” Could someone point me in the right direction?
The model class, for what it’s worth, is the following:
class MultinomialLogitWithEmbeddings(torch.nn.Module):

    def __init__(self, m, n_fe):
        super(MultinomialLogitWithEmbeddings,self).__init__()
        self.n_features = m
        self.n_fe_cats = n_fe        
        self.emb_dim = 1      
        
        # Layers 
        self.emb = torch.nn.Embedding(self.n_fe_cats, self.emb_dim) # Store FEs as an embeddings lookup 
        self.linear = torch.nn.Linear(self.n_features + self.emb_dim, 1, bias=False) # Linear utility layer
        self.lsmax = torch.nn.LogSoftmax(1) # Log soft max activation
        
        # Initialize the layer weights 
        nn.init.kaiming_normal_(self.linear.weight.data)
        
    def forward(self, x_cont, x_cat):    
        x = torch.cat([x_cont.float(), self.emb(x_cat).squeeze(dim=-1).float()], 2)        
        y_pred=self.linear(x)
        y_pred=self.lsmax(y_pred)
        return y_pred.squeeze()