I’ve made a pytorch model which uses a bit of dynamic python such as for looping where the number of loops is specified by entries of a tensor. (It’s similar to a graph neural net. I have since seen pytorch geometric, but it seems that those models can’t be exported to libtorch.) I believe that it is because of this that the model is very slow to train and run. I need to be able to export and use this model in C. I used torch.jit.script and successfully saved the model. I used libtorch to get it to C++ and a header file to make it so I can call the model from C.
The model is very slow to train in python. It was my understanding that a model should become much faster after running torch.jit.script on it (I thought that this turned slow python into fast C++). Would this model train faster if I trained it in C++? Why or why not? What if I just saved the model and reloaded it back into python?
I haven’t included the code here because I couldn’t find an easy way to reduce it to a minimal example that would be removed from the confusing specifics of my research domain. Sorry if this question is hard to grok.
I’m having a hard time understanding why pytorch models are slow or fast and how, in general, they can be made faster. If torchscript makes a dynamic graph static and able to run from C++, then why doesn’t it make a model much faster? (Or does it and I’m using it wrong?)
Thanks in advance,
Jack