Thank you @ptrblck. You are right. The problem came from the CUDA version mismatch. We check our script again and find that we didn’t set the enviroment variable $TORCH_CUDA_ARCH_LIST correctly. This cause the mismatch of CUDA version.  This problem is solved now.