Hi all! I have 4 GPUs 1080 Ti, and when I run training inception_v3 net on multiple GPU model have strange behavior. I didnt rewrite my code much from training on 1 GPU, just add:
model = nn.DataParallel(model, device_ids=[0,1,2,3]).cuda()
When I run script with device_ids=[0,1] GPUs full utilized and train much faster, when I run script with device_ids=[0,1,2] or device_ids=[0,1,2, 3] script starting (GPU full utilized in nvidia-smi, but reserved memory on card small: 1Gb on first card, and 500 Mb on other) and model dont train. Where I am wrong?  And I have 2 cores processor. Thanks for response. Sorry for my English