justusschock:

To tackle this problem, torch.jit traces the dynamically created graph and compiles it into a static version, which can be deployed e.g. via C++ without the need to reimplement the model’s logic.


Perfect feedback @justusschock, I see what you mean. I just found this:

TorchScript is a way to create serializable and optimizable models from PyTorch code. Any code written in TorchScript can be saved from your Python process and loaded in a process where there is no Python dependency.


We provide tools to incrementally transition a model from being a pure Python program to a TorchScript program that can be run independently from Python, for instance, in a standalone C++ program. This makes it possible to train models in PyTorch using familiar tools and then export the model to a production environment where it is not a good idea to run models as Python programs for performance and multi-threading reasons.

(from the official PyTorch documentation 9)
** Now I wonder:
We know that PyTorch on CUDA actually runs on C++. If this is true, then we don’t need to speed up.
One could get the idea if you run PyTorch code (Python) then you are slow. I bet this is not the case. Any clarification would be excellent.