Also, note that

you can use pack_padded_sequence 91 to allow sequences of different length inside a minibatch
it’s perfectly fine to use minibatches with different sequence lengths in every call to LSTM
If your application allows that, using nn.LSTM instead of manually unrolled nn.LSTMCell you can easily observe a 10x speedups.