[trainer] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters
When running DDP:
```
rm -r /tmp/test-clm; CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node 2 \
run_clm.py --model_name_or_path gpt2 --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 \
--do_train --output_dir /tmp/test-clm --per_device_train_batch_size 4 --max_steps 200
```
I get:

> [W reducer.cpp:1050] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters. This flag results in an extra traversal of the autograd graph every iteration, which can adversely affect performance. If your model indeed never has any unused parameters, consider turning this flag off. Note that this warning may be a false positive your model has flow control causing later iterations to have unused parameters. (function operator())


but it's not possible to turn it off from the trainer. i.e. it's hardwired.

@sgugger
Edit: I am not sure why this param is always set to `True` except for gradient checkpointing. We can certainly make a training argument control its value to avoid hard-coding it. At least to experiment and benchmark whether it's best at True/False.