Docs suggest to use discriminator weights for ElectraForMaskedLM instead of generator
## Environment info

- `transformers` version: 4.2.2
- Platform: Linux-5.3.0-64-generic-x86_64-with-Ubuntu-19.10-eoan
- Python version: 3.7.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
 @LysandreJik

## Information

Model I am using (Bert, XLNet ...): ELECTRA

The problem arises when using:
* [x] the official example scripts: https://huggingface.co/transformers/model_doc/electra.html#codecell3
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: I want to check if a some word fits into the context of a sentence, for which I use the prediction probability of that word at the MASK position

## To reproduce

Steps to reproduce the behavior:

1. `from transformers import ElectraForMaskedLM`
2. `model = ElectraForMaskedLM.from_pretrained('google/electra-small-discriminator')`

## My issue

I get the following warning: 

`Some weights of ElectraForMaskedLM were not initialized from the model checkpoint at google/electra-small-discriminator and are newly initialized: ['generator_predictions.LayerNorm.weight', 'generator_predictions.LayerNorm.bias', 'generator_predictions.dense.weight', 'generator_predictions.dense.bias', 'generator_lm_head.weight', 'generator_lm_head.bias']`

I understand that I'm loading the discriminator weights, whereas ElectraForMaskedLM needs the generator weights for the MLM output. Why do the docs tell me to use the discriminator? Am I missing something?



Hello! Indeed, this is a mistake! Do you want to update the docs to show the generator instead?<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.