One possibility is to run the forward once, and use jit tracer to trace the graph. But you may get lower-level ops that implement each layer rather than the layers as individual ops (e.g., instance norm may become: reshape + batch_norm + reshape + mul + add)