Thanks. I would appreciate if you can give a code snippet demonstrating your suggestion. I am aware of  retain_graph, but I don’t see how setting it to True will help accomplish what I asked for. It seems to me that there is still a missing ingredient that would let me to do backward twice on the retained graph, each time keeping a different part of the graph detached.