Solved by RaLo4 in post #2 
              
              
                Can we use .cpu() to save memory on GPU? 

Since I don’t know what you are trying to do here I can’t really answer that question. 
As for this question: 

.cpu() is not inplace for a tensor, so assuming loss is a tensor you need to write it this way: 
loss = loss.cpu()

Also! One reason why a lot o…