I’m not very sure, but I think this maybe because pytorch uses dynamic memory management, something similar to the python automatic garbage collection, to manage and free up GPU memory. Since you have few tensors which occupy a huge chunk of memory, the automatic garbage collection doesn’t kick in.
To make this work, you’ll need to manually delete the output and loss variable at the end of your epoch to free up your GPU memory.