Pegasus Documentation May Conflict With Seq2Seq ReadMe
Here, under `tips and tricks`.....
https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md#tips-and-tricks
`Both finetuning and eval are 30% faster with --fp16. For that you need to install apex.`

But in the documentation...
https://huggingface.co/transformers/master/model_doc/pegasus.html#examples
`FP16 is not supported (help/ideas on this appreciated!).`

Also in the documentation
https://huggingface.co/transformers/master/model_doc/pegasus.html#examples
`Script to fine-tune pegasus on the XSUM dataset.`
leads to a 404: https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune_pegasus_xsum.sh


Hi @kingpalethe,

In general, for BART and Marian models, training and eval is faster with fp16,  except Pegasus and T5 which currently don't work well with fp16

Yes, the fine-tuning script is now moved under `examples/research_projects/seq2seq-distillation` dir,
https://github.com/huggingface/transformers/tree/master/examples/research_projects/seq2seq-distillation

Thanks for reporting,

Also please note that this script is not maintained anymore and is provided as-is. We only maintain the `finetune_trainer.py` script now.<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.