You should keep track of the order somehow. It’s pretty common for seq2seq models to use some kind of attentional input feeding in the decoder which prevents nn.LSTM from being used for all decoder timesteps in one call; in that case I sort the batch based only on source sentence length and use an unrolled LSTM cell in the decoder. Code that sorts the batch together according to the source sentence length will be in torchtext shortly.