I’m experimenting with the idea of generating Torchscript IR as part of a project I’m working on, and would like to better understand how it’s compiled and optimized.
I’ve followed the tutorial here: Loading a TorchScript Model in C++ — PyTorch Tutorials 1.7.1 documentation and successfully loaded a Torchscript model into C++ and ran it on some example data. I’ve also found the overview here: pytorch/OVERVIEW.md at master · pytorch/pytorch · GitHub helpful.
What I’d like to know is:


When I load a module in in C++, is there any compilation / optimization that can performed before the first time I call the forward method? Or do optimizations only happen with a JIT that runs after data is first passed in?


If there are JIT optimizations to be done when I call the forward method, do the optimizations persist so that next time I call the forward method on the same module object, they won’t have to be performed again?


If a module/model gets optimized by the JIT, do any of those optimizations change the output I’d get if I now wrote the model back out to a torchscript .pt file?


Where does kernel fusion happen? Is that part of the JIT, or is that something that has to be figured out when generating the IR in python?


Does Pytorch have the ability to dynamically decide whether an operation should be run on CPU or GPU based on things like data size?


If there’s some resource I haven’t yet seen that could help me better understand this stuff, a pointer towards it would be appreciated. Thanks for your help.