Solved by tom in post #3 
              
              
                The “native” implantations are in 
https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/RNN.cpp and 
https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cuda/RNN.cu 
There also are CuDNN bindings…