[deepspeed integration] HF Trainer takes over GPUs for DP
on multi-gpu machine when `deepspeed` launcher is used w/o the `--num_gpus` arguments to run HF/DS integration, HF Trainer takes over and does DP over all available GPUs, which defeats the purpose of using deepspeed to fit a larger than GPU model.

Normally when no `--num_gpus` argument is passed, deepspeed automatically uses all gpus, so here we have a conflict for those users who are used to that automatic behavior.

So currently we need to always use something like `deepspeed --num_gpus 2` explicitly which tells HF Trainer not use DP over those GPUs.

The proper solution is to code that when `--deepspeed` is passed HF Trainer won't launch DP.

Thanks to Lintang Sutawika for reporting this.
Odd, I can't reproduce this. 

on 2 gpus with `deepspeed --num_gpus 2` or just `deepspeed` I get the same world_size=2 inside Trainer, so it shouldn't make any difference.

So I'm going to close this for now. we can re-open this if someone reproduces the issue.