PyTorch executes eagerly by default. You might be referring to the asynchronous execution of kernel launches on GPUs, where the interpreter will get back to you right away even if the kernel hasn’t finished yet.
There isn’t really a fixed number of warmup iterations, but the reason that they are necessary is that in cases when benchmarking is used to select algorithms (e.g., with torch.backends.cudnn.benchmark=True), the first run will be slower due to the benchmarking overhead. I don’t think there will be any graph-level optimizations without jit: TorchScript — PyTorch 1.9.0 documentation 1