MobileBERT inconsistent output (padded / not padded text)
## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-5.4.0-47-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.3
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik, @mfuntowicz

## Information

When using MobileBERT model `google/mobilebert-uncased`, I get different model outputs for the same text when it contains padding and when it does not contain any padding (is the longest text in the batch). The same code produces correct results for other models, such as `bert-base-uncased`, so the issue is probably only with MobileBERT. The problem happens for both `CLS` output and all features.

Model I am using (Bert, XLNet ...): MobileBERT

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce
```python
from transformers import AutoModel, AutoTokenizer

model = 'google/mobilebert-uncased'
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModel.from_pretrained(model)
text = 'Hey, how are you?'

i1 = tokenizer.batch_encode_plus([text], padding=True, return_tensors='pt')
emb1 = model(**i1)[1][0]  # Only one in batch (not padded): [-2.5088e+07,  7.3279e+04,  1.7544e+05, ...]

i2 = tokenizer.batch_encode_plus([text, text + ' hey'], padding=True, return_tensors='pt')
emb2 = model(**i2)[1][0]  # Not longest (padded): [-2.4871e+07,  8.1873e+04,  1.6693e+05, ...]

i3 = tokenizer.batch_encode_plus([text, text[:-10]], padding=True, return_tensors='pt')
emb3 = model(**i3)[1][0]  # Longest in batch (not padded): [-2.5088e+07,  7.3281e+04,  1.7544e+05, ...]

i4 = tokenizer.encode(text, return_tensors='pt')
emb4 = model(i4)[1][0]  # Without attention masks (not padded): [-2.5088e+07,  7.3279e+04,  1.7544e+05, ...]

i5 = tokenizer.encode_plus(text, return_tensors='pt')
emb5 = model(**i5)[1][0]  # With attention masks (not padded): [-2.5088e+07,  7.3279e+04,  1.7544e+05, ...]
```

## Expected behavior

I would expect the model to always return the same embeddings, but I don't really know which of the two variants is correct.
Just confirming that the issue happens also for the latest `transformers`, PyTorch and Python 3.8.

- `transformers` version: 3.1.0
- Platform: Linux-5.4.0-47-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
