AlbertForMaskedLM always has bad results
I am the one who is using your great project.

I'm trying to make my own ALBERT language model using scratch from [here](https://mlcom.github.io/Create-Language-Model/)
This article seems similar to [your tutorial notebook.](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb#scrollTo=YZ9HSQxAAbme)
I have already made a BERT model that fits my language by referring to this, and it has shown satisfactory results.

```
BertForMaskedLM : about 0.6 loss
BertForSequenceClassification : accuracy 0.88 in my dataset(binary classification)
```

In order to use ALBERT, I trained tokenizer by using Sentencepiece and trained the pretrain model.
My tokenizer had good results, but my language model had higher loss than my BERT model.(about 2.7~2.8)
I thought the result was bad after seeing the loss, so I checked the result with pipeline and fill-mask tasks, and all the results came out the same.

Sentence A
```json
[{'score': 0.7783917188644409, 'token': 32002, 'token_str': '<pad>'},  {'score': 0.008062483742833138, 'token': 3, 'token_str': '.'}, {'score': 0.0054806191474199295, 'token': 4, 'token_str': ','},  ...
```

Sentence B
```json
[{'score': 0.7783915400505066, 'token': 32002, 'token_str': '<pad>'}, {'score': 0.008062485605478287, 'token': 3, 'token_str': '.'}, {'score': 0.005480623338371515, 'token': 4, 'token_str': ','},  ...
```

I want to solve this problem, but I couldn't find the answer even though I found a lot of articles.
I will look forward to the opinions of the contributors and users of this project.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.