I’m running a model where I need to get the gradient of the loss function w.r.t my input data so I can use it to update previous networks that are in series. I thought I was calling everything right, but I have encounted a problem where the gradients of my loss function w.r.t input data is zero or very close to it. What am I doing wrong? The gradients exist throughout the network, loss is nonzero, etc.
Here’s the code for the backpropagation. after_graph is the output of a graph neural network, and net2 is my graph.
after_graph2 = Variable(after_graph.clone(), requires_grad=True)
        
        after_net2 = self.net2(after_graph2)
        # back prop
        loss = self.loss_func(after_net2, y)
        # find gradients of net 2
        loss.backward(retain_graph = True)
        
        loss_graph_output = after_graph2.grad

Here’s the net:
net2 = nn.Sequential(
    nn.Linear(input_size2, hidden_sizes2[0]),
    nn.Tanh(),
    nn.Linear(hidden_sizes2[0], output_size2),
    nn.Tanh()
)

What could be going wrong?