Solved by PeterXiaoGuo in post #20 
              
              
                Hi ptrblck, 
I solve the question posted here by using: 
@voxmenthe ‘s answer from a multiple GPUs’ solution: 
model = <specify model here>
model = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))

I notice you mentioned “it splits the data/batch onto different GPUs” rather th…