DataParallel use quite complex backend methods in CUDA and have some “unpredictable” (by me) behaviour when not used in a classical way.
In particular it uses nccl2 that has some system-wide impact on the gpu.

When the gpu disapear, is your code actually running? If so does it uses multiple gpus or just one?
I am not sure what happens when you do not provide any device_ids to DataParallel by reading the doc 2. I would replace your line by model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3]) and check if that works and properly uses all the 4 gpus. Setting only [0, 1] should use only 2.