Hi @tom,
Thanks for the tips. Will see what I can get by doing that. Its just hard to compare models against LSTMs when they take half a day to train, and given the model I am comparing I would expect it to run faster than LSTMs (forward and backwards operations are simpler). Just one final question (for now ), at the end of the following sentence did you mean CUDA kernel?

Maybe you can get best results on current PyTorch by manually defining a backward in a ,

so that it would be CUDA kernel for the LSTM cell operation and scripted for the forward and backward loop?
Best