Wrong output used by RobertaForSequenceClassification classification head
Hi,

According to the [documentation](https://huggingface.co/transformers/model_doc/roberta.html#transformers.RobertaForSequenceClassification) classification head should work `on top of the pooled output`, which makes sense, considering the fact that RoBERTa, unlike BERT, wasn't trained on Next Sentence Prediction task, so the `<s>` token equivalent to `CLS` is not as useful. However, if you look at the at the code [here](https://github.com/huggingface/transformers/blob/master/src/transformers/models/roberta/modeling_roberta.py#L1166) and [here](https://github.com/huggingface/transformers/blob/master/src/transformers/models/roberta/modeling_roberta.py#L1394) you'll see that it doesn't appear to actually use the pooler output for Sequence Classification, as one would expect. Meanwhile, RobertaForMultipleChoice does use it.

It's not clear to me whether this is intended or not, however RoBERTa using a representation of `<s>` for classification may perform considerably _worse_ than a regular BERT on some tasks.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Hi,

I'd still like to get a comment on whether this is the intended behavior. If it is, then why it is done this way?<|||||>Hello! Sorry for getting back to this so late.

When porting models over to the `transformers` library, we aim to keep them identical to their original implementation. The original RoBERTa implementation in fairseq uses the same classification head, hence why it was ported like this: https://github.com/pytorch/fairseq/blob/c2e8904b6072d8eddab362ac50b324e374b5951d/fairseq/models/roberta/model.py#L382

I recommend opening an issue over at fairseq if you have questions relative to how they designed their architecture. Thank you!<|||||>Ah, I see. Didn't realize it's done the same way in the original implementation. Thank you for your response!