I implemented A3C following the hogwild example:

  
      
      GitHub
  
  
    

ikostrikov/pytorch-a3c 112

PyTorch implementation of Asynchronous Advantage Actor Critic (A3C) from "Asynchronous Methods for Deep Reinforcement Learning". - ikostrikov/pytorch-a3c


  
  
    
    
  
  


It converges on PongDeterministic-v3 in 10 minutes with 16 threads. However, it work poorly on PongDeterministic-v3.
Could you please take a look? I wonder whether I got asynchronous updates in PyTorch right.