Weights used for Masked LM predictions
I wanted to get masked word predictions for a few bert-base models. I am converting the pytorch models to the original bert tf format using [this](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/convert_bert_pytorch_checkpoint_to_original_tf.py) by modifying the code to load BertForPreTraining state_dict. I am unaware of the use of cls/predictions/decoder in the snippet below, to make the masked predictions. The original BERT codebase does not have this layer, hence. Is it used, or can I safely disregard this to obtain predictions?
![image](https://user-images.githubusercontent.com/24687672/104811962-01132c00-5825-11eb-81ef-c7f36c07f457.png)

The `cls/predictions/decoder` is the linear layer that is used to project the output of the transformer to the vocabulary logits. This layer is *tied* to the input embeddings: it has the same weights.

I believe the original BERT codebase doesn't have this layer because it re-uses the input embedding layer, instead of instantiating another one. We do this too in the TF implementation.

You can, therefore, safely disregard this layer is the implementation you're using uses the input embeddings' weights to project the output of the transformer to the vocabulary logits. <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.