Well one thing that seems to work is the following: suppose that device_ids = [1,2]. If I send both the model and the data to device ‘cuda:1’ then the training runs on both gpus. I’m not seeing the speedup I expected but at least it’s running. Also the utilization of gpu 2 is significantly lower than that of gpu 1.  Is this correct? I.e., when using DataParallel do model.to(device) and data.to(device) where device is one of the gpus in device_ids? Or am I still missing something?
Well amend that. It runs but only on two gpus. The training machine has four gpus and when I try the above on 3 or 4 gpus it crashes the machine a few steps after training starts.