Could you try to call DataParallel first, then push the model to the device?
I don’t think it’ll get rid of the error, but it’s the recommended way, if I’m not mistaken.
If that doesn’t help, could you check, if you have to(device) or .cuda() calls inside your model?
If so, these will mess up the DataParallel wrapper, as they force this particular tensor to be pushed to the specified device.