cat/stack is pretty fast, you’re likely measuring asynchronous cuda calls incorrectly. Try the built-in profiler 3. RNNs are optimized (C++), yes. JIT may help you somewhat, but in general recurrent stuff is not fast.