jackeown:

[…] but also that the length of the output may not be known at test time. How does PyTorch deal with this?


That’s a basic issue when it comes to generating sequences such as sentences and is not PyTorch specific. Given a standard RNN-based Seq2Seq model with an encoder and decoder, the decoder generate each next word step by step in a loop. The loop stops if

the next predicted word is an end-of-sequence token, e.g., <EOS>

the number of words is more then a max. threshold (but that’s usually as a safeguard)

This is why you need to add <EOS> as last word to your output sentences when training.
In each run trough the loop the decoder tries to predict the next word, which in turn gives you an error like for any prediction task. And you just sum up those errors over the whole loop.
Note that these simple descriptions assume a batch size of 1. For larger batch sizes, things get a bit more complicated. As @ptrblck mentioned, for training I use the workaround of generating batches with matching input and output lengths. Doing inferences, however, I do with batch sizes of 1.