1. What are you using the PyTorch C++ API for?
Training and generation. We are using C++ because we need to “obfuscate” the code.
*2. How does your software stack look like? And how does the PyTorch C++ API interact with other components in your stack?
Our app/lib is currently available on Linux only using MKL + CUDA + FAISS + RDKit + some other libs. Everything is C++ for the lib, everything is javascript for the client who is using a native NodeJS addon (NAPI used to interact with our lib).
*3. Are you using JIT (TorchScript) models or torch::nn models?
Only torch::nn
*4. Are you training models in C++?
Yes
*5. How do you think the PyTorch C++ API (or the ecosystem in general) should be improved?
Resolve: Static linking master bug #21737
C++17 :  I am stuck with <experimental/ >, I cannot use futures and the likes, constexpr if,… old style lambdas, old style template, …
Thread pools : c10 comes with a ThreadPool but there is no way to wait for job OOB. I think libtorch needs a solid single ThreadPool as creating a threadpool in every module create too many threads.
Documentation : At the very beginning because libtorch was not available for CUDA 10,  I spent a lot of time trying to understand how to compile a compact lib for my platform and my needs without using python: “what USE_THISFLAG is for ? for what purpose? What happen if I set it ON or OFF?” For devs like me who are not at all datascientists, it should be great to introduce the packages you are using (xxblas, xxpack, …).
I am also spending time in converting python <=> C++ (simple example what is torch::mm in python ? what is t.mean() in C++,…)
Thank you