I wonder if this would be helpful:
import torch.multiprocessing
torch.multiprocessing.set_sharing_strategy('file_system')

https://pytorch.org/docs/stable/multiprocessing.html#file-descriptor-file-descriptor 62
  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      Too many open files error 65
    

    
      
        opened 
        
          
        
        Sep 3, 2018
      
      

        
          closed 
        
          
        
        Sep 14, 2018
      
        

      
        
          
          whucdf
        
      
    
  



  Issue description
While using the dataloader from pytorch 0.4.1:
With num_workers > 0 the workers store the tensors in shared memory, but do...



    medium priority (this tag is deprecated)