You can look at the hogwild training example we have:

  
      
      GitHub
  
  
    

pytorch/examples 1.3k

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc. - pytorch/examples