For elementwise kernels, these are typically implemented with PyTorch’s TensorIterator to generate kernels rather than with handwritten CUDA, so there isn’t really any CUDA code to directly inspect in this case.

  
      

      Quansight Labs – 9 Apr 21
  

  
    

PyTorch TensorIterator Internals - 2021 Update 4

  For contributors to the PyTorch codebase, one of the most commonly encountered
C++ classes is
TensorIterator.
TensorIterator offers a standardized way to iterate over elements of
a tensor, automatical