Hi johnc1231,


Not unless you invoke it. Currently the only AOT optimization is torch.jit.freeze — PyTorch master documentation 4. It does a limited set of optimizations on 1.7, a little more on master 2, and I’m working on adding more AOT stuff. I will add a C++ api equivalent by 1.8.


Yes, they do persist, however if you pass in tensors of a different type we may respecialize.


JIT optimizations will not affect saving your model.


That happens as part of the JIT. It’s happens after we profile the tensors that are seen at runtime.


No, that one of the main design points of pytorch is that it does not automatically decide what parts of the module to run on gpu vs cpu, for both eager & jit. That is controlled by the user.


To better understand optimizations I would suggest running a simple file like:
import torch

@torch.jit.script
def foo(x):
    return x + x + x

foo(torch.rand([4, 4], device='cuda'))
foo(torch.rand([4, 4], device='cuda'))

with PYTORCH_JIT_LOG_LEVEL='profiling_graph_executor_impl' foo.py