Thanks for following up on this. I filed an issue to track the problem on GitHub:

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      Deadlock with shared CUDA tensors and multiprocessing (spawn) 109
    

    
      
        opened 
        
          
        
        Jun 4, 2020
      
      


      
        
          
          colesbury
        
      
    
  



  🐛 Bug
Reported by iffiX in https://discuss.pytorch.org/t/freezing-problem-while-using-cuda-tensor-in-multiprocessing-environment/80000.
(I have moved the code into a main() function to rule out some other multiprocessing bugs)
import...





  
  
    
    
  
  


It looks like the deadlock is in CudaIPCSentData destructor, which looks separate from the caching allocator code.