If you’re looking for a fast layer norm LSTM written in CUDA, you can try Haste (https://github.com/lmnt-com/haste 173). I’d love to see how well TorchScript’s performance compares – it would be really nice to have a flexible high-level approach that matches straight-up CUDA code.