Glory_Chen:

For the second problem, maybe it is because of the reason said in Getting Started with Distributed Data Parallel — PyTorch Tutorials 1.7.1 documentation.

DDP processes can be placed on the same machine or across machines, but GPU devices cannot be shared across processes.



Yes, the deadlock is most likely due to this. You should only use a single process per GPU.