What are you using the PyTorch C++ API for?

I’m trying to port the LeelaZero Go AI to use Pytorch as a network inference backend. Hopefully this will enable rapid iteration of new network architectures via JIT models. Currently the network is implemented in OpenCL.

How does your software stack look like? And how does the PyTorch C++ API interact with other components in your stack?

For training and experimentation the Python interface will be used.
Networks are then converted via JIT and run (inference only) in the C++ multi-threaded MCTS engine.
All this is mostly in Linux, but Windows compatibility would be nice too.

Are you using JIT (TorchScript) models or  torch::nn  models?

JIT.

Are you training models in C++?

Not likely.

How do you think the PyTorch C++ API (or the ecosystem in general) should be improved?

First thing that comes to mind, I think the official tutorial would benefit from a short section explaining the way(s) to efficiently transfer between cuda tensors and various cpu data structures, both directions. In general the C++ APIs seem quite under-documented, at least compared to the Python APIs.