albanD:

I don’t think this is what he wants.


True but if in some case he doesn’t plan to backward() multiple times on the same graph, which I think is most likely the case it would be great to do:
backward(create_graph=True, retain_graph=False, ...)

to free the graph.
As per this post 27.