I am facing a similar porblem.
i tried using
model->to(torch::kCUDA)
and
model->to(torch::Device(torch::kCUDA, 0)
but none of them work.
I get the error message
Input type (CUDAFloatType) and weight type (CPUFloatType) should be the same (_convolution at ../../aten/src/ATen/native/Convolution.cpp:599)

any idea how to fix this?