Hi,
I check GPU memory usage by torch.cuda.max_memory_allocated(), and it steadily increases in each forward pass. In the forward pass, it needs to store two intermediate tensors to calculate the next result like Fibonacci. So I think the problem may be in the way that I access these intermediate tensors.
My method:
def forward(self, x):
    ...
    intermediate_result = [] # a tensor list with length of 2, store results of i-1 and i-2
    for i in range(nb_loops):
        next_result = func(intermediate_result) # func is used to calculate next_result
        intermediate_result.pop()
        intermediate_result.append(next_result)
        ...

Could you give me some advice on how to maintain such intermediate_result in forward pass, and how to avoid GPU memory usage increasing problem?
Thanks in advance.