mariosasko:

Sequences are packed to ignore padding


But if all my zeros occur exactly in the end of each sentence, wouldn’t that imply that anything happening during the padding symbols (i.e. anything after the end of sentence) is ignored by the loss function?
In the particular setting I’m describing, I don’t see any difference between the two. One asks to ignore symbols=0, the other asks to ignore symbols after end-of-sentence (which are 0’s)