I’ve created a dedicated library as part of a research for this purpose:

  
      
      GitHub
  
  
    

cheind/pytorch-blender 45

Seamless integration of Blender into pytorch data sources - cheind/pytorch-blender


  
  
    
    
  
  


It creates N-instances of Blender, streaming via ZMQ directly into a PyTorch dataset. The related paper is here https://arxiv.org/abs/1907.01879 37. We used it massively generate Blender renderings for online training of a model. Additionally we created a control channel that allows the simulation to update to training needs.
Work was done for Blender 2.79, but we expect a major leap in performance when switching to 2.80 and EEVEE rendering engine. However, I haven’t looked into image streaming from EEVEE works yet.
Hope that helps.