I have a DataParallel model that has been sent to the GPUs via .to('cuda'). I also have some processes calling this model in parallel at various points. It seems like because these are forward passes of batch size 1, they are automatically allocated to CUDA:0, which results in disproportionately high GPU utilization on that device.
How do I specify which GPU is used in a forward pass? I don’t want to have to do any sending of parameters / state dicts. Thanks.