Fine-tuning Seq2Seq models for Machine translation
Good morning,

@micheledaddetta1

We were experimenting with Seq2Seq models such as MarianMT or T5. 
I was wondering if there is a common way to fine-tune those models with custom datasets for the machine translation task.
Specifically we did the following:

```python
embeddings = self.batch_encode_plus(sentences, padding=True, verbose=False)
embeddings = embeddings.to(self.model.device)

labels = self.batch_encode_plus(target_sentences,  padding=True, verbose=False).input_ids.to(self.model.device)  # expected output tokens
outputs = self.model(input_ids=embeddings.input_ids, labels=decoder_input_ids, return_dict=True)
output_sentences = self.model.generate(**embeddings)
output_sentences = self.decode(output_sentences)
# we also compute sentence embeddings for embedding alignment purposes.
return output_sentences, outputs.loss
```

Is it correct to directly use `outputs.loss` to optimize the model for the machine translation task? 
Maybe @patrickvonplaten or @patil-suraj can chime in here!<|||||>Hey @MorenoLaQuatra 

The `run_seq2seq.py` supports translation with custom dataset. And you can use T5, mT5, MarianMT, mBART, mBART-50 for fine-tuning.
https://github.com/huggingface/transformers/tree/master/examples/seq2seq

And the best place to ask this question is our forum https://discuss.huggingface.co/

Hope this helps :) <|||||>Hi @patil-suraj, thank you for the feedback.

Actually we are trying to train it in a python script (the overall architecture is more complex than the single network). We were asking about the translation loss. If we do as reported in the above snippet, do we optimize the translation loss?
The finetuning for MarianMT use the same head and loss of the training phase?

Thank you for your time.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.