I do not think there is any option/parameter to tell DataParallel which GPUs to use for inference/forward.  If you are only doing inference, won’t it be easy to maintain models in each GPU manually (mode.to(device)) instead of DataParallel?