Hogwild is lock free training
I think what you want if you want use gpu is something like batch-a3c where you have good examples here in pytorch:

  
      
      GitHub
  
  
    

facebookresearch/ELF 60

ELF - An End-To-End, Lightweight and Flexible Platform for Game Research


  
  
    
    
  
  


And tensorflow:

  
      
      GitHub
  
  
    

ppwwyyxx/tensorpack 18

tensorpack - A Neural Net Training Interface on TensorFlow