Solved by tom in post #2 
              
              
                Children are kept in an collections.OderedDict, a Python dictionary to preserve order. 
The nn.Module class does some clever magic around __setattr__ and friends to keep the submodules (and parameters and buffers) in separate dictionaries. 
You can look at the definition, it is clever yet reasonably…