RuntimeError: while running run_common_voice.py (XLSR wav2vec finetuning week)
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.0.dev0 (I tried running it on 4.4.0 as well, gave the same error)
- Platform: Ubuntu (running on a virtual machine)
- Python version: 3.8
- PyTorch version (GPU?): 1.6.0
- Using GPU in script?: yes, running [this script](https://github.com/huggingface/transformers/blob/master/examples/research_projects/wav2vec2/run_common_voice.py)
- Using distributed or parallel set-up in script?: Distributed 

### Who can help
@patrickvonplaten (as per the message on slack group)
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:

- [ ] the official example scripts: (give details below)   
- [ ] my own modified scripts: (give details below)

Tried running both official command and modified script (running command changed based on the language) 

The tasks I am working on is
- [ ]  common voice dataset (ta)

## To reproduce

Steps to reproduce the behavior:

1. run common voice script [from here](https://github.com/huggingface/transformers/blob/master/examples/research_projects/wav2vec2/run_common_voice.py)
2. For multi-gpu setup I used this command `python -m torch.distributed.launch \
	--nproc_per_node 4 run_common_voice.py \
	--model_name_or_path="facebook/wav2vec2-large-xlsr-53" \
	--dataset_config_name="tr" \ # use this argument to specify the language code
	--output_dir=./wav2vec2-large-xlsr-turkish-demo \
	--overwrite_output_dir \
	--num_train_epochs="5" \
	--per_device_train_batch_size="16" \
	--learning_rate="3e-4" \
	--warmup_steps="500" \
	--evaluation_strategy="steps" \
	--save_steps="400" \
	--eval_steps="400" \
	--logging_steps="400" \
	--save_total_limit="3" \
	--freeze_feature_extractor \
	--feat_proj_dropout="0.0" \
	--layerdrop="0.1" \
	--gradient_checkpointing \
	--fp16 \
	--group_by_length \
	--do_train --do_eval `

## Error:
`RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument 'find_unused_parameters=True' to 'torch.nn.parallel.DistributedDataParallel'; (2) making sure all 'forward' function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's 'forward' function. Please include the loss function and the structure of the return value of 'forward' of your module when reporting this issue (e.g. list, dict, iterable).`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Model would train without any error
<!-- A clear and concise description of what you would expect to happen. -->

I am experiencing the same error. I have been all the day working around without solving it. I have tried trough Docker Cuda versions 9.2, 10.0 and 10.1 and different versions of pytorch including 1.3, 1.5 and 1.6. I have tried with different combinations of GTX1080 and RTX2080

Adding "--ddp_find_unused_parameters=true" to the python command does not fix the error.

Any help is really appreciated as I am working on the fine-tuning week @patrickvonplaten <|||||>I am experience this error too.
CUDA 11.2
4xT4 - 16Gb
`--dataset_config_name="ru"`<|||||>@raja1196 I think I have found the bug. Could you try modifying in run_common_voice.py the gradient_checkpointing to False, as it is written below:

```
gradient_checkpointing: Optional[bool] = field(
        default=False,
        metadata={
            "help": "If True, use gradient checkpointing to save memory at the expense of slower backward pass."
        },
    )
```

And then running the script without gradient_checkpointing as follows:

`python -m torch.distributed.launch \ --nproc_per_node 4 run_common_voice.py \ --model_name_or_path="facebook/wav2vec2-large-xlsr-53" \ --dataset_config_name="tr" \ # use this argument to specify the language code --output_dir=./wav2vec2-large-xlsr-turkish-demo \ --overwrite_output_dir \ --num_train_epochs="5" \ --per_device_train_batch_size="16" \ --learning_rate="3e-4" \ --warmup_steps="500" \ --evaluation_strategy="steps" \ --save_steps="400" \ --eval_steps="400" \ --logging_steps="400" \ --save_total_limit="3" \ --freeze_feature_extractor \ --feat_proj_dropout="0.0" \ --layerdrop="0.1" \ --fp16 \ --group_by_length \ --do_train --do_eval`

This solves the problem in my case and now I am able to run it with two GPUs. If it works to you, I will do PR<|||||>@ivangtorre s solution works. unfortunately, I have to reduce the batchsize quite a lot.

Update: I stopped using distributed training for now, as I did not get any performance gains somehow. Does anyone know whether the CTC loss of this model is computed in a distributed way, or are the outputs gathered on a single gpu before computing loss?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.