Hi,
I am implementing a GNN, which uses Chebyshev’s polynomials. I have a for loop that uses the recursive relation between Chebyshev’s polynomials:
# for k = 0, T_0(L) = I
Tx = T0_x = graphs.clone()
    
if self.cheby_K > 1: # T_1(L) = L
    T1_x = laplacians @ graphs
    Tx = torch.cat((T0_x, T1_x),dim=-1)
    # T_k(x) = 2*L*T_{k-1}(x) - T_{k-2}(x)
for k in range(2, self.cheby_K):
   Tk_x = 2 * laplacians @ self.T1_x - T0_x
   Tx = torch.cat((Tx,Tk_x),dim=-1)
   T1_x, T0_x = Tk_x, T1_x

When I test my network with the following snippet:
with torch.autograd.set_detect_anomaly(True):
        with torch.enable_grad():
            
            graphs = torch.unsqueeze(nodes,0).contiguous().to(device)
            
            laplacians = torch.unsqueeze(laplacian,0).to(device)
                
            for i in range(0,100):
                
                optimizer.zero_grad()
                
                logits, out_nodes, out_laplacians = gnn(graphs,laplacians)
                
                loss = criterion(logits,labels)
                
                print('epoch:{1:d}, loss: {0:.4f}'.format(loss,i))
                
                probs = torch.softmax(logits,dim=-1)
                
                print('predictions:\n',probs.type(torch.float16).data.numpy())
                
                loss.backward()
                
                optimizer.step()

I receive the following error message

Blockquote “RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time.”

According to the error, I must specify the retain_graph=True when calling .backward in my loop above, but the training code again raises another RunTimeError:

Blockquote RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [8, 32]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I dont use any hidden state. Also, if I set self.cheby_K=1, the training runs without an error.
Could you help me to figure out what I am doing wrong?
Thanks
Edit:Typo