If you need to use CUDA and c++ have you tried to use the Nvidia runtime TensorRT?
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html 1
https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html
Or you can find an example using directly libtensorflow at:
  

      Medium – 12 Oct 20
  

  
    

Using the new Tensorflow 2.x C++ API for object detection (inference) 1

  Serving Tensorflow Object Detection models in C++

  
    Reading time: 3 min read
  

  

  
    
    
  

  


/cc @markdaoust  do you know if we have any other inference example on GPU and Windows? As we have already another request at:
  
    
    
    Basiс things to deploy the model on windows C++ in Visual Studio General Discussion
  
  
    Hi, I have a model (autoencoder) it works well in python. Now I need to embed it into the existing c++ project on Windows, 
After reading the site, I haven’t yet the basic plan of how to do It. Do I have to Install TF2 with basel and use TF2 C++API ? I can’t find anything similar to .predict in python. Or do I have to use c_api? Can somebody explain to me the common practice? 
Do I have to Use c_api, or C++ api? Or I have to use TFLite for inference? 
Thanks!