Bort always predict wrong words
Even on api interface Bort predict wrong words in MaskedLM for example here - https://huggingface.co/amazon/bort?text=Paris+is+the+%3Cmask%3E+of+France.

So there has to be some configuration bug that should be fixed.

The same happens when calling Bort using auto classes.

Could you take a look?


Pinging @stefan-it here as he contributed this model<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Hi @Oxi84 , we were not able to get mask prediction working, even with the original Gluonnlp implementation. So I think it's a good idea to disable the inference for now.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.