Solved by tom in post #2 
              
              
                I’m not sure anything in your code immediately jumps into the eye that is bad. 
So there a couple of things to keep in mind here: 


PyTorch has a caching GPU memory allocator, this means that it will not return memory to the system when freeing tensors but instead keep it and re-use it for the next…