When you do backward pass, some of the values of the graph for the intermediate (non-leaf) nodes are freed. So, second pass over same nodes will trigger this error. To be able to proceed with second pass, you need to add parameter to loss1.backward(retain_graph=True)