For libtorch models, we also need to use model->to(torch::kCUDA) to convert the model to CUDA. See https://github.com/pytorch/pytorch/blob/505fa83b2f4b02b55784c94ba52d4ab41d351623/test/cpp/api/modules.cpp#L295-L310 9.