The issue is raised, since the intermediate tensors are already freed (after you’ve used retain_graph=True), while your backward() call tries to backpropagate through operations where these intermedates are already deleted. Detaching the tensor would solve the problem (the backward pass would stop at this point and will not backpropagate further), but I understand that it might not be trivial if the code base is complicated.