If I’m not mistaken, the default JIT backend uses up to 3 steps to optimize the graph for each new input shape. I.e. the new and larger batch would trigger the optimization again. The nvfuser backend would relax this slowdown a bit, as it’s more flexible when it comes to different input shapes (in case the generated kernels cannot be reused, they would be regenerated and optimized again).