Yeah it’s the second situation you mentioned, I actually append the model output to a list because the loss calculation need three times forward and I use this list to collect three times forward output, and then use torch.cat to concatenate the list to a Tensor, the loss is calculated based on this Tensor. But I actually call the ‘del loss’, couldn’t this step clear the calculation graph?