+1 to @osalpekar’s comment.
One thing I want to add is that, when DDP was initially introduced, it has two modes:

single-process multi-device (SPMD): each process exclusively works on multiple GPUs, and hence there will be multiple model replicas within the same process. In this case, the device_ids should be a list of GPUs one process should use.
single-process single-device (SPSD): each process exclusively works on one GPU, i.e., each process works on a single model replica. In this case, device_ids should only contain a single device.

As SPSD is almost always the recommended way to use DDP due to perf reasons, we are planning to retire SPMD mode soon. If there are concerns, please comment here: https://github.com/pytorch/pytorch/issues/47012 1