Sure, D



 lzkzls:

Thank you so much! When I created 4 DDP instances in four different processes using argument process_group, this error disappeared. And could you give me some explanation about the affecting of precess_group？ Why we should create different processes for different DDP instances?


Sure. When creating DDP instances in the same process without specifying process_group argument, those DDP instances will share the same default process group. If you use these DDP instances interleavingly, it’s likely to cause communication de-synchronization, which would lead to hang or crash (see this brief explanation of DDP implementation). So if you would like to create multiple DDP instances on the same process for different models, you can try provide different process group instances.  You can create new groups using the new_group API.