jackeown:

The model is very slow to train in python. It was my understanding that a model should become much faster after running torch.jit.script on it (I thought that this turned slow python into fast C++).


Note that both, the Python and C++ frontend, call into the same backend methods.
In the usual case you would run your model on the GPU, so that Python is used for the model “logic” and the dispatching of the kernels (to C++ and CUDA).
For “big” workloads the Python overhead might be marginal.



 jackeown:

If torchscript makes a dynamic graph static and able to run from C++, then why doesn’t it make a model much faster?


If you have a static graph (with your conditions etc.), then some optimization might be applied such as fusing certain operations together.
However, it depends on your actual model, if fusing ops is possible. Also note, that this area is still being worked on.