Solved by tom in post #2 
              
              
                The short answer is that you cannot easily do this without doing the backward for each separately, i.e. 
grads1 = torch.autograd.grad(loss1(output), model.parameters(), retain_graph=True)
grads2 = torch.autograd.grad(loss2(output), model.parameters())

This gives you two lists of gradients, ordered …