Hi @ptrblck ,
Thank you for taking the time to read my post and reply! You are right that a particular batch was larger than others and it caused the program to crash there. To resolve that I curbed the max length of my generated sequences.
However, when I tracked the tensors in my program with the debug_memory snippet, it does seem like the intermediate tensors are still kept until I compute the loss for the next batch, which I believe is strange?
But with regards to the memory issue, you are entirely spot on! Thank you for your help!