Is that the complete code or do you create any logs etc. after the optimization?
Usually you will run out of memory, if you store a tensor with its computation graph. E.g. if you use total_loss += loss instead of total_loss += loss.item().