Hi @melody-ju
Not sure what you mean by  "T5 doesn’t work out-of-the-box for seq2seq tasks ". T5 is a seq2seq model and it does work for seq2seq tasks.
You can use Trainer for seq2seq tasks as it is. Patrick’s PR extends it so that generative metrics can be calculated (ROUGE, BLUE etc), it should be okay if you calculate them after training the training is finished.
To use Trainer for T5, the dataset or collator (if you are using one) should at least return
input_ids, attention_mask and labels (set pad tokens to -100 in labels). The rest will be handled by Trainer
This notebook 33 uses Trainer for fine-tuning T5.
Few things to note about  that notebook,
I wrote it before v3.0.0, few things have changed after that


DatCollator is not a class anymore, so you won’t need to inherit from DataCollator when creating T2TDataCollator. Also collate_batch should be renamed to __call__.

lm_lables is now deprecated, use labels instead.
No need to manually add </s> anymore, the tokenizer  now does that itself.

Also you can use the prepare_seq2seq_batch method on toknizer which can take the source and target text and returns input_ids,  attention_mask and labels.
You can also use finetune.py script from here 12 to finetune T5 and other seq2seq models. It’s using PL, and there;s WIP version of Seq2SeqTrainer in this PR 5