Thanks @valhalla for your explanation.
To confirm my understanding.


Resizing the embedding will add extra rows/columns for the new tokens, which is initialised with random weights, correct ?


Seq2Seq example:
https://github.com/huggingface/transformers/blob/master/examples/seq2seq/seq2seq_trainer.py#L119 38
Will use teacher forcing during training, is there anyway to disable teacher forcing in the library, or I have to implement it my self by feeding the model one output at a time sequentially ?