That’s not correct. On a single machine with multiple GPUs you often want to opt for DDP, too, as it performs much better than DP. You can check the official pytorch documentation for this if you’re interested, but if you can, I always advise to use DDP for true parallelism (true multiprocessing rather than single process multithreading with GIL slow down).
Alse see: https://pytorch.org/tutorials/intermediate/ddp_tutorial.html 6