Hm, that’s tricky then.
What you could try is to set output_device=torch.device('cpu') in DataParallel, but I am not sure if this will work and how fast this will be. I’d probably rather go with a smaller batch size instead of doing that.
Another thing that I don’t know if it would work is e.g., using device ids for cuda 0-2 via device_ids and then setting output_device=torch.device('cuda:3'). Let me know if that works, I’d be curious.