If you are using the native lstm, it is normal that the native one is faster than touchscript. Also, according to here 58, increasing the batch size can reduce the difference in performance. And according to my own experience, increasing the hidden size can also reduce the difference.