Your example uses nn.DataParallel and therefore is not related to distributed. It also doesn’t use multiprocessing. I recommend changing the title to something like PyTorch nn.DataParallel hang or similar to better convey what’s going on.
When the process hangs, can you acquire any more information, such as where it is hanging? Can you attach a debugger and get a stack trace, for example? Or identify the exact line in your Python script where the hang occurs?