I traced some classification models from torchvision and then wanted to apply  DataParallel for multi-gpu inference. While no error is thrown, for more than 1 gpu passed as device_ids to DataParallel, inference on the torchscript model is much slower than the regular torch model (5x-10x slower), even if batch_size=1, which results in effectively only 1 gpu being used.