Thank you for your answer.
I was told that the JIT compiler of PyTorch makes the for-loop overhead negligible. Is this true?
Packed sequence is weird. You need to first pack your padded sequences before passing them to the RNN and then unpack the result in order to use it for something else such as word tagging. Isn’t that a large performance overhead apart from the extra memory needed for storing the packed sequence?
Also, it seems that the embedding function of PyTorch does not work the same way that Keras does, that is, it doesn’t include a mask together with the embedded words. Does this mean that the packed sequence approach is the only way to handle variable length sequences in PyTorch RNNs or is there more than one way? If that is the only way, then are there tools available to help you make a custom RNN that makes use of packed sequences?