Thank you! Your explaination is very clear and is very helpful! I still have a contains that I followed your suggestion and trained a summarization model using nn.TransformerDecoder (I used BERT as an encoder), the training seems go well (training can generate meaningful sentences), but the inference process only generate the same token (repeatly). I am wondering does the training and inference different in settings? (The only difference I did for training and inference is that in the inference, I send one token at the very begining and add the generated token to the decoder for the next prediction.)