You’d have to comment out entire if input.is_cuda section for your modifications to take effect, otherwise, a hardcoded fused kernel is called that calculated the standard lstm cell. If performance in inadequate after that, you might want to look at creating your own extension with the LSTM cell architecture you want. https://github.com/pytorch/tutorials/pull/214/files 47