@apaszke, as @ThibaultGROUEIX  said,  it’s better to mention this in pytorch document. This also waste me much time before I find this post.
BTW, I highly recommend this full tutorial example pytorch-custom-cuda-tutorial 91