Inconsistency in how Electra doing sentence level prediction
In `ElectraForSequenceClassification`:
We have docstring says, `ELECTRA Model transformer with a sequence classification/regression head on top (a linear layer on top of
    the pooled output) e.g. for GLUE tasks.`
This is also what I observed in the official repository.
https://github.com/google-research/electra/blob/81f7e5fc98b0ad8bfd20b641aa8bc9e6ac00c8eb/finetune/classification/classification_tasks.py#L270
https://github.com/google-research/electra/blob/79111328070e491b287c307906701ebc61091eb2/model/modeling.py#L254
which is
```
nn.Sequential( nn.Dropout(config.hidden_dropout_prob),
                        nn.Linear(config.hidden_size, config.num_labels)
```

**But** the implementation of `ElectraClassificationHead` (used by `ElectraForSequenceClassification`) is
```
def forward(self, features, **kwargs):
        x = features[:, 0, :]  # take <s> token (equiv. to [CLS])
        x = self.dropout(x)
        x = self.dense(x)
        x = get_activation("gelu")(x)  # although BERT uses tanh here, it seems Electra authors used gelu here
        x = self.dropout(x)
        x = self.out_proj(x)
        return x
```

Is there something I overlooked in the official repository ? Hot to explain the inconsistency in doc and implementaion of `ElectraForSequenceClassification`.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Are there any updates on this issue?