I also implement SRU in pytorch，I have two version, one is that use the author code by call the SRU, another is that implement it by formula, this is my demo, https://github.com/bamtercelboo/pytorch_SRU 104, II dont know that yours whether very correctly and even my code I also dont know, we can communicate each other, I have a question in cuda, SRU by formula is not speed up compare to the CPU, I don`t know the reason, you konw?