Hi ptrblck,
I solve the question posted here by using:
@voxmenthe ‘s answer from a multiple GPUs’ solution:
model = <specify model here>
model = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))

I notice you mentioned “it splits the data/batch onto different GPUs” rather than model sharding… I feel puzzled on this statement. What’s the advantage to split model class on different GPUs?  Does it mean it helps to distribute model training burden to multiple PCs (rather one PC only?)
Thank you very much!