You can use trainer for seq2seq as well, you’ll just need to write a different data collator which will return the expected arguments to the model.
Few things to note about the that notebook,
I wrote it before v3.0.0, few things have changed after that


DatCollator is not a class anymore, so you won’t need to inherit from DataCollator when creating T2TDataCollator. Also collate_batch should be renamed to __call__.

lm_lables is now deprecated, use labels instead.

Let me know if you run into problems using the notebook.