Only init encoder_attention_mask if stack is decoder
We currently initialize `encoder_attention_mask` when it is `None`,
whether the stack is that of an encoder or a decoder. Since this
may lead to bugs that are difficult to tracks down later, I added a condition
that assesses whether the current stack is a decoder.
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=h1) Report
> Merging [#1770](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/1c542df7e554a2014051dd09becf60f157fed524?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `90%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1770/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1770      +/-   ##
==========================================
+ Coverage   84.03%   84.03%   +<.01%     
==========================================
  Files          94       94              
  Lines       14032    14034       +2     
==========================================
+ Hits        11792    11794       +2     
  Misses       2240     2240
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/1770/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2JlcnQucHk=) | `87.82% <90%> (+0.04%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=footer). Last update [1c542df...cd286c2](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Ok great @rlouf!<|||||>I wonder why encoder_attention_mask use decoder inputs shape when it's not specified. 
`encoder_attention_mask = torch.ones(input_shape, device=device)`<|||||>Indeed, wanna have a look @rlouf?<|||||>That's a very good point, thanks @efeiefei ! See #2107