I have two 1080Ti GPUs.
If I train my network on one GPU, everything seems to work fine. I use something like:
dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=200, shuffle=True, num_workers=1,pin_memory=True) for x in ['train', 'val','test']}

device = torch.device("cuda:1" if torch.cuda.is_available() else "cpu")

model_ft.to(device)


When I use the same model, and the same data wrapped in Dataparallel, the system crashes in the first epoch.
My code is:
dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=200, shuffle=True, num_workers=1,pin_memory=True) for x in ['train', 'val','test']}

if torch.cuda.device_count() > 1:
          device = torch.device("cuda")
          print("Let's use", torch.cuda.device_count(), "GPUs!")
          model_ft = nn.DataParallel(model_ft)


model_ft.to(device)

I can make the Dataparallel work if I reduce the batch size to something as small as 40. And I don’t feel this speeds up my training.
Any thoughts?
Thanks!
S