It is a problem due to the version of cuda, cudnn. It is solved after I installed a new cuda and cudnn (cuda 9.0 and cudnn 7). I guess currently pytorch cpp library can only work on some version. It is not stable.