n_gpus is set to 1 in case of distributed training on multiple gpus, how to access to the correct n_gpus
Hi
I am using transformer 4.1.1 on multiple gpus with distributed launch script below, the n_gpu set in this case is 1 instead of actual n_gpus, could you tell me how I can access to the total number of ranks calling the launcher with? Is there any variable in huggingface library which could show the number of gpus the script is called with? Here is the command I run: 

`export BS=4; CUDA_VISIBLE_DEVICES=0,1  USE_TF=0   python -m torch.distributed.launch --nproc_per_node=2 --master_port=9915  finetune_trainer.py --model_name_or_path t5-small --output_dir output_dir --adam_eps 1e-06 --data_dir wmt_en_ro --do_train --freeze_embeds --label_smoothing 0.1 --learning_rate 3e-5 --logging_first_step --logging_steps 1000 --max_source_length 128 --max_target_length 128 --num_train_epochs 1 --overwrite_output_dir --per_device_train_batch_size $BS --sortish_sampler  --task translation --val_max_target_length 128 --warmup_steps 500 --n_train 500
`

thanks 
Please do not spam the issues: this is a duplicate of #9225. You can edit the title and the description, there is no need to send a new notification to everyone watching the repo.
Also, please use the [forum](https://discuss.huggingface.co/) for questions like this, we keep the issues for bugs and features requests only. In your case, this is a PyTorch command: `toch.distributed.get_world_size()` will return you the number of GPUs used.<|||||>Hi Slvain, thank you, sorry I did not really realized the duplicate, sure, thanks 