Bug in GPT2ForSequenceClassification
## Environment info
google collab

##Problem
When I run the example code provided on huggingface site for GPT2ForSequenceClassification, an error is raised sayin that 'microsoft/dialogrpt' is not a checkpoint. See the bug bellow:
![bug1](https://user-images.githubusercontent.com/48957493/115378658-10619700-a1d1-11eb-949b-67c46735534c.png)

Then , I replace 'microsoft/dialogrpt' by 'gpt2' but when I run the code twice, the logits have diffetent values at each run. After i've dug deeper, I've seen that the problem occurs when the top linear layer is being built. Its weights seem to be set randomly, so they differ from a run to another. Do you have any way to get rid of this problem? Thanks!


Ah, indeed, I think the identifier is incorrect. It needs to be updated to `microsoft/DialogRPT-updown`. Do you want to open a PR to fix this?

The issue you mention regarding the random weights is because the `gpt2` checkpoint doesn't have a sequence classification head. This is not the case for the checkpoint mentioned above (`microsoft/DialogRPT-updown`) which does have a sequence classification head, so it will not be reinitialized every different run.<|||||>@LysandreJik Thanks a lot! Indeed, the identifier `microsoft/DialogRPT-updown` works! But, can I change the code example on hugging face site through a pull request? Since I won't really change some source code, is it a change that must be made in the Transformers repo? The (wrong) code example is as following
![bug](https://user-images.githubusercontent.com/48957493/115460610-c3111400-a228-11eb-8ad0-3db5e9d1ac44.png)

<|||||>Yes, the example is created here: https://github.com/huggingface/transformers/blob/5e04d7086803ae4a3892f4082f2835a756592c2c/src/transformers/models/gpt2/modeling_gpt2.py#L1239

If you can update this, then it will update it in the docs as soon as we merge it.

All the docs visible on the website are in the repository :)<|||||>Ok, thanks! I'll deal with it then :)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.