Adding GPT2 for translation/summarization
# 🚀 Feature request

Hi
In the code base GPT-2 is not listed as the model compatible for summarization and translation. Is there a reason for this?

## Motivation
GPT-2 is one of the most important seq2seq models and many papers build on top of it. It would be great to include this model. 

Thanks. 
The `run_summarization.py` and `run_translation` scripts are designed for encoder-decoder (also called seq2seq) models like T5, BART, PEGASUS. 

However, GPT-2 is a Transformer decoder-only model. This model is also capable of doing summarization/translation, but it would be solved in a different way. GPT-2 is just a language model (i.e. it predicts the next token given the previous ones). So you could fine-tune GPT-2 using the [`run_clm.py`](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling) script. OpenAI has done this for example in [this paper]().

What typically happens is one provides the text, appended by "TDLR: " to the model, and the model is then capable of generating a summary (just because it is trained on a large portion of the internet).

<|||||>I was actually looking for this to do some Legal Text Summarization. Thanks Niels for explaining it well. Looks like I need to do it on a different model. Do you have any model to suggest that works well with legal text?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.