Thanks for your suggestion. I tried
module = torch::jit::load("traced_model.pt", torch::kCUDA);
module.to(torch::kCUDA);

but got the same results. I have the debug dlls and library etc, perfectly ready for some more debugging. Anything more I can do to help?
I’m stepping through line-by-line. I noticed that the module.forward() call takes about 18 seconds before the exception and this happens even when I know I’m giving it a wrongly sized Tensor:
torchinputs.push_back(torch::ones({1, 1, 1, 1}, torch::kCUDA).to(torch::kFloat)); // intentionally wrong size
module.forward(torchinputs);

If I change everything in my code to cpu, it doesn’t throw a runtime error. So I must be not succeeding in making sure everything is CUDA. I also tried following everything here https://github.com/pytorch/pytorch/issues/19302 4
Why isn’t this sufficient for having everything in CUDA?
auto module = torch::jit::load("traced_model.pt", torch::kCUDA);
for (auto p : module.parameters()) {
	std::cout << p.device() << std::endl; // cuda:0
}

auto finalinput = torch::ones({ 1, 3, 320, 320 }, torch::TensorOptions().dtype(torch::kFloat).device(torch::kCUDA));
std::cout << "finalinput device: " << finalinput.device() << std::endl; // cuda:0
torchinputs.push_back(finalinput);
auto forward_result = module.forward(torchinputs); // std::runtime_error

^ and changing merely both of those two references to kCPU instead of kCUDA doesn’t throw an error.