output padding different to zero in hidden layers with attention mask
# 🐛 Bug
On the last layer, the token corresponding to the padding does not return 0, even if attention masking is used.
## Information

Model I am using (Bert, XLNet ...):
roberta and roberta XLM

Language I am using the model on (English, Chinese ...):
english

The problem arises when using:
* [x] the official example scripts: (give details below)


The tasks I am working on is:
looking at the output of the last layer of RobertaModel

## To reproduce

Steps to reproduce the behavior:

1. use some padding in your inputs data
2. create accordingly the attention mask
3.  

example of code :
```

def tokenize_sentences_Bert(sentences, tokenizer, maxlen):
    tokens = []
    lengths = []
    for s in sentences:
        token = tokenizer.encode(s, add_special_tokens=True, max_length=maxlen)
        lengths.append(len(token))
        token = token + [tokenizer.pad_token_id]  * (maxlen - len(token) )

        tokens.append(token)

    return tokens, lengths
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
model = RobertaModel.from_pretrained('roberta-base',
                                          output_hidden_states=False,
                                          output_attentions=True)
max_length = 10
sequence = ["I eat a green apple", "I am playing football tomorrow"]
tokens, lengths = tokenize_sentences_Bert(sequence, tokenizer, maxlen=max_length)

lengths = torch.tensor(lengths)
tokens = torch.tensor(tokens)
attention_mask = (torch.arange(max_length).expand(len(lengths), max_length) < lengths.unsqueeze(1)).float()
print(attention_mask)
outputs = model(tokens, attention_mask=attention_mask)
print(outputs[0][:, :, :2]) # last step should return 0 last hidden layers
```



- `transformers` version:
- Platform: ubuntu
- Python version: 3.6
- PyTorch version (GPU?): 1.4
- Tensorflow version (GPU?):
- Using GPU in script? no:
- Using distributed or parallel set-up in script?: no

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
