Ah, whoops I really need to get more sleep =)
Yes, in the kaggle code, I’ve explicitly put batch_first=True now.
In layman terms, the network is trying to train to generate a sequence of word.
E.g. given the sentence: <s> this is a foo bar sentence </s>
X: <s> this is a foo bar sentence
Y: this is a foo bar sentence </s>
Let me check the output size again.