GPT Neo for Sequence Classification
Hi,
Is there a way to use GPT Neo for classification tasks like BoolQ ? 
As 'OpenAI GPT2' integration of HF has 'GPT2ForSequenceClassification', is there a similar one for GPT Neo?
@patil-suraj this may be a good first issue? Feel free to open a PR!<|||||>Thanks @NielsRogge .

Hi @patil-suraj ,

Is there any workaround to make it work in my local?<|||||>We could for sure add `GPTNeoForSequenceClassification`. 

It would be as easy as 
- just copying the `GPT2ForSequenceClassification` module and replacing the `GPT2` with `GPTNeo`
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/src/transformers/models/gpt2/modeling_gpt2.py#L1225-L1231
- `config.hidden_size` instead of  config.n_embd
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/src/transformers/models/gpt2/modeling_gpt2.py#L1232
- remove the model_parallel logic
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/src/transformers/models/gpt2/modeling_gpt2.py#L1236-L1238
- add a test in `tests/test_modeling_gpt_neo.py`  similar to
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/tests/test_modeling_gpt2.py#L357

Marking this as "Good First Issue".

Feel free to take a stab if you want, I would be happy to help.<|||||>Hi Guys,
is anyone working on this?
I can make PR for this. I might also need to use it in the future.
<|||||>Hi @bhadreshpsavani, Feel free to open a PR :) 