Solved by ptrblck in post #5 
              
              
                The easiest way would be to lower your batch size. If that’s not possible (e.g. if your batch size is already 1), you could have a look at torch.utils.checkpoint to trade compute for memory.