Solved by ptrblck in post #3 
              
              
                I’m not sure how your use case works exactly, but 

to keep the computation graph (and the intermediate tensors) alive, you could use backward(retain_graph=True).