I think, the gradient calculations of next iterations won’t be affected because every forward call would be working on their own set of data. But if the computational graphs are not cleaned up, you would eventually face the Out of memory error such as:

  
    
    
    CPU RAM explodes after 30 minibatch iterations 
  
  
    I have read over 10 different reports of a similar problem with exploding RAM on a CPU and none of them have worked. 
I can run my forward pass using just the encoder network without any problems so I know my DataLoader and Encoder are fine. If I use the Decoder just for forward passes, without even storing or computing any losses, the RAM explodes. 
Therefore, what is it about my decoder that causes this? I tried to follow this https://pytorch.org/tutorials/intermediate/seq2seq_translation_tuto…