Transformer-XL not working with DistributedDataParallel
# 🐛 Bug

## Information
I'm using the pretrained `TransfoXLLMHeadModel` for finetuning but I get an error when trying to use it with pytorch's `DistributedDataParallel`:
> self.reducer.prepare_for_backward(list(_find_tensors(output)))
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward` function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable). (prepare_for_backward at /opt/conda/conda-bld/pytorch_1579022034529/work/torch/csrc/distributed/c10d/reducer.cpp:514)

I'm using the pytorch distributed launch utility.  
I've already gone through similar issues regarding this topic, but none of the solutions solved my problem. A guess from me is, that this is caused by the returned `mems` and `DistributedDataParallel` could think that they require a gradient?

Model I am using: Transformer-XL

Language I am using the model on: English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)


## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-108-generic-x86_64-with-debian-buster-sid
- Python version: 3.6.10
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes `torch.distributed.launch`

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I have the same kind of problem with transformers 3.3.1 and a BertModel, but the error is now an inplace operation on torch.cuda.LongTensor problem, something related to EmbeddingBackward.<|||||>Could you please open a new issue with the full stacktrace + the information related to your environment?<|||||>Done as issue #7848.