You can actually perform training with torchscript.
You can try to torchscript your python code during training.
That being said, if your network is a regular architecture, we try to make sure that the performance for these are as good as possible out of the box.