Reformer self attention mask value not converted in apex half precision
# 🐛 Bug

## Information

Model I am using Reformer:

Language I am using the model on English:

The problem arises when using:
* [ x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Install the latest transformers 2.9.0 and apex

```
from transformers import ReformerModel, ReformerTokenizer
import torch
from apex import amp

tokenizer = ReformerTokenizer.from_pretrained('google/reformer-crime-and-punishment')
model =  ReformerModel.from_pretrained('google/reformer-crime-and-punishment').cuda()

optimizer = torch.optim.Adam(model.parameters(), lr=0.0001)

model, optimizer = amp.initialize(model, optimizer, opt_level='O1')

input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0).cuda()  # Batch size 1
outputs = model(input_ids)

last_hidden_states = outputs[0]  # The last hidden-state is the first element of the output tuple

```

## Expected behavior

```
File "/home/model/env/lib/python3.7/site-packages/transformers/modeling_reformer.py", line 840, in forward
    query_key_dots = torch.where(mask, query_key_dots, mask_value)
RuntimeError: expected scalar type Half but found Float
```


- `transformers` version:
- Platform: Ubuntu 18.04
- Python version: 3.7
- PyTorch version (GPU?): 1.5.0 cuda
- Tensorflow version (GPU?): No tensorflow
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=h1) Report
> Merging [#4242](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/7b75aa9fa55bee577e2c7403301ed31103125a35&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4242/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4242   +/-   ##
=======================================
  Coverage   78.39%   78.39%           
=======================================
  Files         120      120           
  Lines       19925    19925           
=======================================
+ Hits        15620    15621    +1     
+ Misses       4305     4304    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/4242/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yZWZvcm1lci5weQ==) | `88.17% <0.00%> (ø)` | |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4242/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.93% <0.00%> (ø)` | |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4242/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `74.38% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=footer). Last update [7b75aa9...a14259c](https://codecov.io/gh/huggingface/transformers/pull/4242?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
