Solved by fmassa in post #4 
              
              
                If you want to recursively iterate over modules, then you want to use .modules() 
For example, in the following network 
m = nn.Sequential(nn.Linear(2,2), 
                  nn.ReLU(),
                  nn.Sequential(nn.Sigmoid(), nn.ReLU()))

calling m.children() will return 
[Linear (2 -> 2), ReLU…