I define the subnet as a Independent network and instantiate it in the main net like this:
Class SubNet(nn.Module):
    def __init__(self):
        ...
    def forward(self, inputs):
        ...

Class MainNet(nn.Module):
    def __init__(self):
        ...
        self.subnet = SubNet()
    def forward(self, inputs):
        ...
        output = self.subnet(feature_map)
        ...

and the optimizer looks like this:
mainnet = MainNet()
optimizer = optim.SGD(mainnet.parameters(), lr=0.01, momentum=0.9)

I just print the parameters number of the main net, I notice that the number changed if I add a subnet in the main net. Does this means that the optimizer will optimize all the parameters in main net and subnet?
Thanks for all the reply.