Disparitry with Fairseq Roberta implementation for predicting the mask token
## ❓ Questions & Help
When experimenting with the fill mask functionality on the fairseq repo I realized there is a disparity with the results I get from huggingface implementation. Wondering if there is a mismatch between the model released here and their latest release.

Thanks
We would need more information<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Also cc @joeddav <|||||>Ok, I've reproduced an issue here.

```python
sentence = ' My favorite type of cheese is <mask>!'
roberta_fairseq = torch.hub.load('pytorch/fairseq', 'roberta.base')
roberta_hf = pipeline('fill-mask', model='roberta-base')

roberta_fairseq.fill_mask(sentence)
"""[(' My favorite type of cheese is goat!', 0.10264863073825836, ' goat'),
    (' My favorite type of cheese is cream!', 0.07233648002147675, ' cream'),
    (' My favorite type of cheese is broccoli!', 0.057516228407621384,' broccoli'),
    (' My favorite type of cheese is bacon!', 0.037444233894348145, ' bacon'),
    (' My favorite type of cheese is ham!', 0.03281955048441887, ' ham')]"""

roberta_hf(sentence)
"""[{'sequence': '<s> My favorite type of cheese is goat!</s>',
     'score': 0.09398240596055984,
     'token': 24791},
    {'sequence': '<s> My favorite type of cheese is cream!</s>',
     'score': 0.07240654528141022,
     'token': 6353},
    {'sequence': '<s> My favorite type of cheese is broccoli!</s>',
     'score': 0.06303773820400238,
     'token': 34803},
    {'sequence': '<s> My favorite type of cheese is bacon!</s>',
     'score': 0.04124978929758072,
     'token': 18599},
    {'sequence': '<s> My favorite type of cheese is jack!</s>',
     'score': 0.03125162795186043,
     'token': 10267}]"""
```

They don't align when I use roberta large either. cc @sshleifer.<|||||>Probably want to compare the encoded input IDs. If they are the same, then there is a model difference (which is a big deal). Otherwise it can be attributed to a different tokenisation (e.g. difference in spaces or difference in adding special tokens).<|||||>The input IDs do match (again, when an empty space is prepended in fairseq's case): `[    0,  1308,  2674,  1907,     9,  7134,    16, 50264,   328,     2]`. 

I doesn't look like a pipelines issue:

```python
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
roberta_hf = RobertaForMaskedLM.from_pretrained('roberta-base')
roberta_fairseq = torch.hub.load('pytorch/fairseq', 'roberta.base')
roberta_fairseq.model.eval()

sequence = 'My favorite type of cheese is gouda!'
tokens = torch.tensor([tokenizer.encode(sequence)])
hf_out = roberta_hf.forward(tokens)[0]
fairseq_out = roberta_fairseq.model.forward(tokens)[0]

torch.mean(torch.abs(fairseq_out - hf_out)).item()
# 0.053489409387111664
```

Masked LM outputs differ by an average of `0.053`.<|||||>I don't have any time to look into this further, but from looking at the named parameters of both implementations, it seems that there is a difference in the LM head, that is, everything that comes after the encode (embedding + 12 layers). I can't pinpoint it exactly, but it seems note-worthy that the weight LM head in fairseq, comes from the token embedding weights

https://github.com/pytorch/fairseq/blob/4923f34790761f41170fd88cd06e4d00ab0c527c/fairseq/models/roberta/model.py#L286-L291

I am not sure in how far that is the same in transformers, but it seemed peculiar. Particularly, it seems that `transformers` does the 'None' case here, whereas it should actually take the token embedding weights.

https://github.com/pytorch/fairseq/blob/4923f34790761f41170fd88cd06e4d00ab0c527c/fairseq/models/roberta/model.py#L211-L213

I might be completely wrong, though. Other input welcome.

**Tl;dr**

Transformers does this

```python
self.decoder = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
```

whereas fairseq does

```python
self.lm_head = RobertaLMHead(
    embed_dim=args.encoder_embed_dim,
    output_dim=len(dictionary),
    activation_fn=args.activation_fn,
    weight=self.sentence_encoder.embed_tokens.weight,
)
...
# `weight` is **not** None
if weight is None:
    weight = nn.Linear(embed_dim, output_dim, bias=False).weight
self.weight = weight
```

I'd be very interested to know _why_ fairseq does this. Why would you want specifically those weights to be the same? <|||||>I implemented my findings in https://github.com/huggingface/transformers/pull/2928.

The following snippet (based on the problem stated above) runs as expected, with identical results for both `transformers` and `fairseq`.

```python
import torch
from transformers import RobertaTokenizer, RobertaForMaskedLM, pipeline

# init HuggingFace
tokenizer_hf = RobertaTokenizer.from_pretrained('roberta-base')
roberta_hf = RobertaForMaskedLM.from_pretrained('roberta-base')
# init fairseq
roberta_fairseq = torch.hub.load('pytorch/fairseq', 'roberta.base')
roberta_fairseq.model.eval()

# LM test
sequence = 'My favorite type of cheese is gouda!'
tokens = torch.tensor([tokenizer_hf.encode(sequence)])
hf_out = roberta_hf.forward(tokens)[0]
fairseq_out = roberta_fairseq.model.forward(tokens)[0]
print(torch.mean(torch.abs(fairseq_out - hf_out)).item())
# should be 0.0

# ---
# pipeline test fill-mask
sentence = ' My favorite type of cheese is <mask>!'
print(roberta_fairseq.fill_mask(sentence))
roberta_hf = pipeline('fill-mask', tokenizer=tokenizer_hf, model='roberta-base')
print(roberta_hf(sentence))
# should have identical predictions
```<|||||>Resolved by #2958