As explained by the author of this question, the execution order is not maintained by using .named_children() and modules will be returned based on the initialization. I don’t know if there is a clean way to perform this type of model surgery automatically without e.g. overriding some modules manually.