can't set gradient_checkpointing=True when using the DistributedDataParallel mode
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.15.0
- Platform: 2GPU-V100 with 16GB
- Python version: 3.6.8
- PyTorch version (GPU?): 1.10.1+cu102
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:

### Who can help

@sgugger 

I used the official document to build a pretrain BigbirdMLM model in DP mode, the trainer works well (but when the seq_length is very big ~4096, the training would take a lot of time), so I would like to try the DDP mode, but I meet some problems:
1. Do I need to change some codes in the original main.py? I read the document of "trainer_ags.py", so I only take a code change from "ddp_find_unused_parameters=False" to "ddp_find_unused_parameters=True", but raise a error about the "gradient_checkpointing", so I change the "gradient_checkpointing=False" ( this parameter is set to True in DP mode), and then I run the code in command line with "torchrun --nproc_per_node=2 main.py", it works well. 
2. the feature of "gradient_checkpointing" is very important I think. If I set "gradient_checkpointing=False", the batchsize(per_device_train_batch_size) only can be set to 1, (otherwise OOM), so this feature has a conflict in DDP mode? sorry I don't know the details about this feature. thanks



You need to have `dpp_find_unused_parameters=False` when using `gradient_checkpointing=True`.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.