Pre-trained electra-large model doesn't converge when fine-tuned on SST-2
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): ELECTRA 

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: SST-2


## To reproduce

Steps to reproduce the behavior:

1. Load the large pre-trained ELECTRA model using ElectraModel.from_pretrained('google/electra-large-discriminator', output_hidden_states=True)
2. fine-tune it on SST-2 using a simple binary classification head (linear, ReLU, linear, Sigmoid) on top of the [CLS] hidden state with BCEWithLogitsLoss and AdamW for 3-4 epochs.
3. Model is stuck on ~55% accuracy during all training and the loss increases

Important: When I do the same with 'google/electra-base-discriminator', and 'google/electra-small-discriminator' I'm getting an accuracy of ~93% and 89% (respectively) on the first epoch.

Hyperparameters: 
batch_size: 16
lr: 0.000005
adam_epsilon: 0.0000001
max_len: 32

## Expected behavior

I expected that the fine-tuned electra-large model will outperform the base and small electra model and have better results than ~50% accuracy

## Environment info
   
- `transformers` version: 2.9.0
- Platform: Linux-5.3.0-1017-aws-x86_64-with-debian-buster-sid
- Python version: 3.6.10
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


This is a general question and does not sound like a bug. With these large models it is sometimes hard to find a good set of hyperparameters to get the model to converge well. The same is true, and reported, for ALBERT. I don't think this is a bug.<|||||>@shanybarhom A good start would be to use the hyper-parameters mentioned in the [ELECTRA](https://arxiv.org/abs/2003.10555) paper :) Just refer to table 7.

So your batch size, adam epsilon and epochs are very different compared to the ELECTRA parameters.<|||||>Thanks, @stefan-it I've tried to use the same hyper-parameters as mentioned in the ELECTRA paper (lr=0.00005, batch_size=32, adam_epsilon=0.000001, epochs=3), but the electra-large model still doesn't converge (accuracy of ~50%). 

@BramVanroy  I thought it is a bug since the electra-base and the electra-small converge quite quickly (90% accuracy after the first epoch) with the same code and data, while the large model is stuck on ~50% during the training, so it felt like a bug, but of course, it may not.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I am facing the same problem with ELECTRA-large. I really appreciate if there is any further direction for this.