Hello Everyone,
I am new to Pytorch and I wanted create some novel architecture.

jump_arch.PNG719×268 10 KB

I wanted to create something like the second architecture so I try to save the tensors at every block inside a list like [i, c, p, fc, o] so that I can use the calculated tensors for a new connection. But first, I have a problem when I try to do .backward()
My forward function goes like this
def forward(self, inputs):
        tensor_list.append(inputs)
        for i in range(len(self.nnModList)):
            x = self.NNList[i] (tensor_list[i])
            tensor_list.append(x)

and loss.backward(retain_graph=True)
                or

def forward(self, inputs):
        tensor_list.append(inputs)
        for i in range(len(self.NNList)):
            x = self.NNList[i] (tensor_list[i])
            tensor_list.append(x.detach())

It works but does not seem to converge unlike doing the forward without the list. How can I make the code above work like the normal?