There was an PR to provide such a feature for general Python objects, but not landed yet. You can copy that code for now.

  
      github.com/pytorch/pytorch
  
  
    
  
    
  

  
    
      [distributed] implement all_gather for arbitrary python objects
    

    
      pytorch:gh/rohan-varma/25/base ← pytorch:gh/rohan-varma/25/head
    

    
      
        opened 
        
          
        
        Oct 28, 2019
      
      

      
        
          
          rohan-varma
        
      

      
        
          +93
          -0