I want to use pytorch for multi-gpu parallel computation. I see that the official documentation offers the following way:
device = torch.device("cuda:0")
model = nn.DataParallel(model)
model.to(device)

I am a bit confused since if we are sending the model to “cuda:0”, how does the other GPUs are going to be utilized?