I use 32GB memory GPU to train the gpt2-xl and find every time I call the backward(), the memory will increase about 10GB. So the training will stop after 2 epochs because the memory    use out. I try some methods like call the torch.cuda.empty_cache() or ‘del loss, output’ after optimizer.step() but it seems not work well. I wanna know it is because I didn’t delete the calculation graph of last epoch correctly?