Fix bug of multi-gpu training in lm finetuning
Current code will raise error when running multi-gpu training (n_gpu > 1 & local_rank = -1).
 

# [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040?src=pr&el=h1) Report
> Merging [#1040](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/pytorch-transformers/commit/9d0029e215f5ad0836d6be87458aab5142783af4?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #1040   +/-   ##
=======================================
  Coverage   79.55%   79.55%           
=======================================
  Files          42       42           
  Lines        6863     6863           
=======================================
  Hits         5460     5460           
  Misses       1403     1403
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040?src=pr&el=footer). Last update [9d0029e...856a63d](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1040?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Make sense, thanks @FeiWang96!