del loss would eventually delete the loss after the gradients were already calculated, but your use case would still store 3 full computation graphs (with their intermediate activations which are needed to compute the gradients), so I assume you are trying to delete the loss after the loss.backward() was called? If so, wouldn’t the OOM error be already raised at this point?