Thanks for the reply,
I assume the differences in speed are for the LSTM cell, so that doing the processing over longer sequences with the full LSTM architecture will increase the difference, especially during training?