Hi, I am trying to run a pytorch trained model following the instructions here: https://pytorch.org/tutorials/advanced/cpp_export.html 17
I want to run the model in the GPU if I run:
    torch::DeviceType device_type;
    if (torch::cuda::is_available()) {
        std::cout << "Cuda available, running on GPU" << std::endl;
        device_type = torch::kCUDA;
    } else {
        std::cout << "Cuda NOT available, running on CPU" << std::endl;
        device_type = torch::kCPU;
    }
    torch::Device device(device_type);
    module->to(torch::Device(device));

I get the message: Cuda available, running on GPU. Nevertheless, when I execute:
at::Tensor output = module->forward(inputs).toTensor();
The command consumes way to much time and I can see at the system monitor how all cores are being used and how no new GPU process is opened in the nvidia-smi tool.
I guess I am doing something wrong but I can’t find what. Any ideas?
I am using Cuda 9.0 and built the project from libcudnn7_7.4.2.24-1
I imported:
#include <torch/script.h> 
#include <torch/cuda.h>

Thanks a lot and sorry for the inconvenience.