Hi @berkayberabi
You are right, in general, Trainer can be used to train almost any library model including seq2seq.
Seq2SeqTrainer is a subclass of Trainer and provides the following additional features.

lets you use SortishSampler

lets you compute generative metrics such as BLEU, ROUGE, etc by doing generation inside the evaluation loop.

The reason to add this as a separate class is that for calculating generative metrics we need to do generation using the .generate method in the predict step which is different from how other models to prediction, to support this you need to override the prediction related methods such as (prediction_step,  predict) to customize the behaviour,  hence the Seq2SeqTrainer.
Hope this answers your question.