Thanks for your input @dgriff!
However, this tutorial addresses the DQN algorithm in opposition to A3C, I am trying to implement. The issue I ran into is that I can’t call render(mode='rgb_array') in different processes when using torch.multiprocessing  in order to spawn multiple worker agents and asynchronously update the global model as described by Mnih et al. (https://arxiv.org/pdf/1602.01783.pdf 2).