thanks for reply; so if your interpretation is correct; gr_a and gr_b is actually been treated as a function here and differentiation of each one w.r.t. neural network parameters does happen and the update of the parameters do consider gr_a and gr_b as loss func.
But with retain_graph = True in the first round of backward()
change the second loss to var_loss = torch.mean(gr_a+gr_b) still gives the same error.
So I guess retain_graph=True in the first round doesn’t really play too much here?