Hey @khalil, sorry about the delay. Looking at the screenshot, they are actually the same set of processes, but each process used all GPUs.  Any reason for setting device_ids to None, instead of device_ids=[local_rank]? If it is None, by default, DDP will use all visible CUDA devices.