Thanks a lot for the feedback!
I’m seeing that you are quite experienced with distributed system; Do you know how much the optimizer cancel each other e.g. If one optimizer does a step and is reset, does it also cancel the accumulation of the other optimizers ? I saw a thread about that and I am quite curious about it
Also, what do you think is the best way to retrieve the global network parameters:
using .share_memory() and retrieve it from time to time, or using a multiprocessing.Manager() like in this implementation 2 ?
mp_manager = mp.Manager()
shared_state = mp_manager.dict()
shared_state["Q_network_state_dict"] = q_network.state_dict()