DistributedDataParallel for multi-gpu single-node runs in run_lm_finetuning.py
# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
Modify `run_lm_finetuning.py` with DDP for multi-gpu single-node jobs.


## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
In it's current state, `run_lm_finetuning.py` does not run in DDP for multi-gpu single-node training jobs. This results in all but the first GPU having very low utilization (as low as 50%, when the first one is in the high 80%) due to the way simple DP works. Once implemented, the load would be more evenly balanced across all the GPUs.
## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->
I can help implementing this feature, but would need guidance on what should/shouldn't be modified to get this working properly.
As far as I can see, the script fully supports DDP:

https://github.com/huggingface/transformers/blob/2ba147ecffa28e5a4f96eebd09dcd642117dedae/examples/run_lm_finetuning.py#L282-L286

I haven't run the script myself, but looking at the source this should work with the [torch launch](https://pytorch.org/docs/stable/distributed.html#launch-utility) utility. Your command would then look like this when using a single node with four GPUs.

```bash
python -m torch.distributed.launch --nproc_per_node 4 run_lm_finetuning.py [arguments]
```
<|||||>Ah ok, I wasn't aware that it had to be launched this way. I was looking at the code and thought it DDP would happen only when the process was launched across multiple nodes.

Thanks for the help @BramVanroy 