With backward(retain_graph=True) I can keep the current graph for future backprops. I understand that the last backprop then should have retain_graph=False in order to free the graph. However, at the point of the backward pass I do not have this information (yet). Therefore, is there any way to manually free the graph? (Hopefully, other than detaching each Variable or running a backward without update?)