Profiling the cell I find  that only 20% of the computational time is spent on the matrix multiplication. On the other hand 70% is spent on tanh, sigmoid, add, mul, cat and slice (roughly equally divided between each of them). Does it come from the kernel launch latency mentioned earlier in the conversation by @apaszke? If yes is there any future plans/hope to improve on this? Thanks.