Solved by vainaijr in post #2 
              
              
                do you mean something like this, 
class outer_network(nn.Module):
  def __init__(self):
    super().__init__()
    self.fc = nn.Linear(10, 20)
    self.inner_network_1 = nn.ModuleDict([['inner_fc', nn.Linear(20, 30)]])
    self.inner_network_2 = nn.ModuleDict([['inner_fc', nn.Linear(20, 30)]])

mode…