Solved by ptrblck in post #2 
              
              
                Are you sure it’s a memory leak? 
It seems you are passing cas_input in a “recurrent” way into your modules, such that its computation graph will be stored. If you don’t want to calculate the gradients for cas_inputs, you could call .detach() on it before passing it to self.D_convs[i].