Hi everyone,
I can understand the theoritical concept of the centralized learning-decentralized execution approach, but I am quite confused about the coding-engineering changes to be done in the update of the networks in the PPO algo.
I think that the actor network (I have seperate networks) will use each agent’s actor loss to update the network, but how the critcs are updated? Should I calculate the cummulative critic loss (from all the agents) and backpropagate it in every single critic network?