Enhance a MarianMT pretrained model from HuggingFace with more training data
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**: https://stackoverflow.com/questions/63774619/enhance-a-marianmt-pretrained-model-from-huggingface-with-more-training-data
Have you tried the finetune.sh script shown [here](https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune.sh)? In addition to the short list of CLI flags listed there, you could try adding:

```
--src_lang "en" \
--tgt_lang "de" \
--num_train_epochs 400 \
--warmup_steps 20 \
--train_batch_size 32 \
--eval_batch_size 32 \
--data_dir "/data/dir" \
--output_dir "/path/to/store/model/etc" \
--cache_dir "/path/for/misc/files" \
--max_source_length 128 \
--max_target_length 128 \
--val_max_target_length 128 \
--test_max_target_length 128 \
--model_name_or_path "</path/to/pretrained>"
```

where the "/path/to/pretrained" could be either a local path on your machine or MarianMT model (Opus-en-de or equivalent). The "data/dir" has a "train.source" and "train.target" for the source & target languages, such that line number x of the target is a translation of line x in the source (and same with "val.source" and "val.target"). I have changed the finetune.py script [here](https://github.com/huggingface/transformers/blob/77cd0e13d2d09f60d2f6d8fb8b08f493d7ca51fe/examples/seq2seq/finetune.py#L415) to 
```
parser = TranslationModule.add_model_specific_args(parser, os.getcwd())

```
and then ran the finetune.sh script.


Note: The gradients blew up when I used the "fp16" flag (with Pytorch 1.6), so I had removed it. Also, you might want to check on the "val_check_interval", "check_val_every_n_epoch", and probably check [this issue](https://github.com/huggingface/transformers/issues/3447) on how to save multiple checkpoints.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
