Yes, I was able to solve the issue. I found my input tensors have different size. So the DataParallel couldn’t properly split the input tensors into GPUs. So maybe check if your inputs size is consistent?