Hi,
In general the multiprocessing setup looks good to me.
It looks like on the link you mentioned, the authors have implemented ShareAdam: https://github.com/g6ling/Reinforcement-Learning-Pytorch-Cartpole/blob/master/parallel/1-Async-Q-Learning/shared_adam.py 14 to share gradients across processes. If your use case requires this too, this is probably a good approach, as PyTorch does not currently natively support sharing gradients across processes in optimizers.