If you don’t use distributed, you can simply do model = nn.DataParallel(model, gpus=list_gpus_to_use).
If you want to use distributed, I don’t know as I’ve never done it   But the doc 101 should contain all the details you need. Inparticular for init_process_group, it states " This blocks until all processes have joined.", hence the hanging you were seeing.