I’ve just found the reason that the program must be used on cuda:0. I’m sorry that I used cuda(1) actually as 0 was used by others then but I changed it to 0 here.
Now when I changed it to cuda(0), everything goes right.
But it is still very strange for me why the cuda kernel doesn’t implement at cudas except cuda(0)?  As our GPU usage is of high frequency, it is of importance that these pytorch cuda extensions can be used on any GPU, so how can it be implemented on other GPUs?
I’m sorry that I forgot to say the versions:
OS: ubuntu 16.04 LTS
python: 3.6.8
pytorch: 1.0.0