Thank you!
I’m start run_common_voice.py 17 with torch.distributed.launch and all 4 GPU the loading normal.
I had to set the size per_device_train_batch_size="3", otherwise the system would give an error CUDA: out_of_memory.  Why? What is the acceleration from this use of multiGPU, in fact, the size of the total batch is 12?  Where am I going wrong?