How to use pack_padded_sequence in seq2seq models
Then when I sort sequences by length I get: So, h(hidden state) and c(cell state) from encoder will be incorrect for corresponding sentences in the decoder in the batch. How can I solve it? Or I shouldn’t use pack_padded_sequence? Thanks!
VladislavPrh
You should keep track of the order somehow. It’s pretty common for seq2seq models to use some kind of attentional input feeding in the decoder which prevents nn.LSTM from being used for all decoder timesteps in one call; in that case I sort the batch based only on source sentence length and use an unrolled LSTM cell in the decoder. Code that sorts the batch together according to the source sentence length will be in torchtext shortly.
jekbradbury
VladislavPrh
jekbradbury
vijendra_rana
After attempting to implement seq2seq with packed_sequences, I have come to the conclusion that this is not really feasable as, It appears to me that fundamentally, packed sequences require a lot of fenangling to work as currently implemented.  It feels like they could benefit from a little more abstraction to make things a bit easier for the programmer. Happy to be corrected/proven wrong here!
DuaneNielsen