I am currently trying to understand the actor critic example 24 for the cart-pole environment.
I understand the general principle and how the algorithm works. However in this code we have a neural net with two heads. One output for our actions and another one for our predicted future reward based on the current state.
My question is about these few lines of code:
# reset gradients
    optimizer.zero_grad()

    # sum up all the values of policy_losses and value_losses
    loss = torch.stack(policy_losses).sum() + torch.stack(value_losses).sum()

    # perform backprop
    loss.backward()
    optimizer.step()

    # reset rewards and action buffer
    del model.rewards[:]
    del model.saved_actions[:]

Here we are adding the policy loss and the value loss. But as i understood it, we should call backward once for the loss of the policy and once for the value loss? Why is it sufficient here to just add them up?
Thanks in advance