Hello Wen-Yi,
In addition to the wonderful advice provided by @KFrank, I’ll add the following:
You will find that people deploy ML models in a variety of ways, but an approach preferred by many is to train models in Python and run inference in C++. Libtorch has a really nice C++ API and you can write code to construct and train models in C++, as well as to run inference. In my environment, I use C++ for training as well as for inference.
If you do take the C++ route, I would encourage you to compile Libtorch from source and ensure that your LD_LIBRARY_PATH/DYLD_LIBRARY_PATH is set to point to the correct location of your Libtorch libraries. Some folks prefer statically linking, but I always link dynamically, taking care to ensure that my LD_LIBRARY_PATH is correctly set. One thing that CMake will need is to know how to find your Libtorch libraries, and the CMakelists.txt file in the tutorial (https://pytorch.org/tutorials/advanced/cpp_frontend.html 8) shows how to accomplish that.
Once you get the base build going, you can move onto building with CUDA/ROCM if you want acceleration.
The C++ tutorial is nicely written and should provide you all you need to get going. In the event you really do embrace the C++ route, I would suggest that you took a look at Conan (http://conan.io 5), which is a popular technique to build, package, and deploy C++ libraries. You can look at my github repo (github.com/forwardmeasure 9) for the Conan recipe for Libtorch.
HTH,
PN