Update thread, following this thread:

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      possible deadlock in dataloader 7
    

    
      
        opened 
        
          
        
        Apr 25, 2017
      
      

        
          closed 
        
          
        
        Oct 9, 2018
      
        

      
        
          
          zym1010
        
      
    
  



  the bug is described at pytorch/examples#148. I just wonder if this is a bug in PyTorch itself, as the example code...





  
  
    
    
  
  


It seems if I put too complicated logic like parsing JSON or iterate over list in  __get_item__, it will fail, if I just put a simple string split logic, it will pass. same as some one mentioned in github thread
so here comes my question, if I can not put much complicated data process in data_loader, should I have another multi processing logic in our main train.py and put some data cook job there to leverage the multi process in pytorch? what is best proactice
neverthless, I think that github bug should not be closed, issue is still there