Hi @GuillaumeZ, I should actually update my previous comment - the example I linked to is outdated.
I suggest to read the following blog post: Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models
It includes a good overview (as well as links to notebooks) on how to fine-tune warm-started encoder-decoder models using the Seq2SeqTrainer (which is an extension of the Trainer).
Note that one still needs to define the decoder_input_ids himself when using a decoder like BertLMHeadModel or RobertaLMHeadModel. This will be updated in a PR I’m currently working on, such that the decoder_input_ids will be created automatically based on the labels provided by the user.