I think the idea is that one would re-use the CUDAExtension.
There is a patch regarding CUDAExtensions on ROCm (it seems to be stalled but you could use locally, probably, and it would seem to be a relatively future-proof way) :

  
      github.com/pytorch/pytorch
  
  
    
  
    
  

  
    
      [ROCm] Initial port of CUDAExtensions to HIP 10
    

    
      pytorch:master ← iotamudelta:feature_extensions
    

    
      
        opened 
        
          
        
        Jun 21, 2019
      
      

      
        
          
          iotamudelta
        
      

      
        
          +22
          -8
        
      
    

  

  
  
    
    
  
  


Best regards
Thomas