Multi-node training with the latest transformers/examples code
Hi,

I am trying to follow the instructions on how to use the examples from [https://huggingface.co/transformers/examples.html] and I notice there is a difference between version 4.3 and 1.2 in the distributed training session. In the older version it seems that it supports multi-node training with 
"    --node_rank=$THIS_MACHINE_INDEX \
    --master_addr="192.168.1.1" \
    --master_port=1234 run_bert_classifier.py \"
  But these options no longer exist in the latest tutorial. Does the latest version still support multi-node training? Thanks.
cc @sgugger <|||||>I'm unsure what you think is not supported. Launching any of the example scripts with 
```
python -m torch.distributed.launch --nproc_per_node=xxx \
   --node_rank=$THIS_MACHINE_INDEX \
   --master_addr="192.168.1.1" \
   --master_port=1234 \
   run_xxx.py
```
is going to work.<|||||>Hi @sgugger , thank you for your answer. My understanding is that "--nproc_per_node" is the number of gpus will be used for the launched process? Also, if I want to launch another training node, I assume I will just run the same command with "node_rank=1"?<|||||>Yes, that is the number of GPUs.

You can refer to the [PyTorch documentation](https://pytorch.org/docs/stable/distributed.html#launch-utility) for all the arguments of the PyTorch launcher as all the example scripts are fully compatible with it. You will also need to pass `--nnodes=$NUMBER_OF_NODES` for completeness.<|||||>Thanks, I am able to make it work now.