Hi,
I am currently working 3D deep learning with 3D convolutions on grids. Now, I encountered the problem that if the 3D grid size is above (110, 110, 110) the number of allocated tensors on the GPU starts to grow when calling the loss.backward() causing a GPU memory leak.
Why is that the case?
An example:
    x1 = self.padding(x)
    x1 = self.kernel_1(x1)
    x1 = self.tanh(x1)

    x1 = torch.cat((x, x1), dim=1)
    
    x1 = self.kernel_2(x1)
    x1 = self.tanh(x1)

    x1 = self.kernel_3(x1)
    x1 = self.tanh(x1)

    x1 = self.kernel_4(x1)
    x1 = self.tanh(x1)

    x1 = self.kernel5(x1)
    x1 = self.tanh(x1)

    return x1

If I return x1 after kernel_3, then the problem does not happen. If I return x1 after kernel_4 or later the problem happens. All kernels are 3D convolutions with 1x1x1 kernels.
Thanks in advance!