You cannot do forward and backward passes in separate processes.
However, have you looked at just having the models themselves whole-ly running in separate processes?
For example: Have you looked at our Hogwild example?

  
      
      GitHub
  
  
    

pytorch/examples 2

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc. - pytorch/examples


  
  
    
    
  
  


Or https://github.com/ikostrikov/pytorch-a3c 9