You should use the @torch.jit.script annotation, which will recover the full semantics of your model (including any control flow that depends on input size). See the tutorial for more info: https://pytorch.org/tutorials/advanced/cpp_export.html 39