Failure to load checkpoints saved during distributed training
# 🐛 Bug

## Information

Model I am using: Bert

Language I am using the model on: English, German, Swedish

The problem arises when using:
* [x] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run [run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) using distributed training: e.g. 
`python3 -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 run_language_modeling.py` --output_dir output_dir [--args...]
2. When the training is over, try to load the final model: BertModel.from_pretrained('output_dir'): this works.
3. Then, try to load a checkpoint: e.g., `BertModel.from_pretrained('output_dir/checkpoint-1000')`: this gives a runtime error:
`
Traceback (most recent call last):
  File "/cluster/shared/nlpl/software/modules/in5550/202002/lib/python3.7/site-packages/transformers/modeling_utils.py", line 470, in from_pretrained
    state_dict = torch.load(resolved_archive_file, map_location="cpu")
  File "/cluster/shared/nlpl/software/modules/pytorch/1.4.0/lib/python3.7/site-packages/torch/serialization.py", line 529, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/cluster/shared/nlpl/software/modules/pytorch/1.4.0/lib/python3.7/site-packages/torch/serialization.py", line 709, in _legacy_load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: storage has wrong size: expected 4434893008627221919 got 2359296
`

## Expected behavior
We should be able to load from the checkpoints also when the training is distributed.

## Suggested solution
Check `if args.local_rank == -1 or torch.distributed.get_rank() == 0` on line 370 (just like on line 736).

## Environment info
- `transformers` version: 2.5.0
- Platform: UNIX
- Python version: Python 3.5.3
- PyTorch version (GPU?): 1.4.0 
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Distributed (1 node, 4 GPUs)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
