Latest Wav2Vec2 pretraining script runs on first GPU only
### System Info

Dear @patrickvonplaten 

I've tried different multi GPU setups (RTX 3090 and A5000) but training always runs only on device 0.

**Tried these commands:**

`accelerate launch --num_processes=2 pretrain.py`
`accelerate launch pretrain.py`

Seems there is some bug in the script because old fairseq training runs on all available devices.
Kindly assist to understand where to dig further.

**Script used for training:**
[https://github.com/huggingface/transformers/blob/main/examples/pytorch/speech-pretraining/run_wav2vec2_pretraining_no_trainer.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/speech-pretraining/run_wav2vec2_pretraining_no_trainer.py)

### Who can help?

@patr

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Just followed the instruction on the script page

### Expected behavior

Run on multiple GPUs
Got to run accelerate config in order to setup devices.