Solved by Brando_Miranda in post #62 
              
              
                If I understand correctly what you should do to run on multiple GPUs is for all GPUs 
net = torch.nn.DataParallel(model, device_ids=list(range(torch.cuda.device_count())))

if you want to use a set of specific ones: 
net = torch.nn.DataParallel(model, device_ids=[0,1,2,5,10,...])


Note: you actuall…