Hi, I am just a beginner of PyTorch, even a beginner of Python. So this question may too simple for you but I really want to get some help.
I want to train a neural network which is included sub-networks. It looks something like that.

nnp.png468×559 72.1 KB

In my opinion, this network could be implemented by constructing three simple fully-connection neural networks.
My implementation is:
#firstly, define the simple NN
class Net(nn.Module):
    def __init__(self,n_input,n_hidden1,n_hidden2):
        super(Net, self).__init__()
        self.hidden1 = nn.Linear(n_input,n_hidden1)
        self.hidden2 = nn.Linear(n_hidden1,n_hidden2)
        self.output  = nn.Linear(n_hidden2,1)

    def forward(self,x):
    x = F.tanh(self.hidden1(x))
    x = F.tanh(self.hidden2(x))
    x = self.output(x)
    return x

#construct 3 same nets
net = []
for i in range(3):
    net.append(Net(3,5,5))


# define loss function and optimizer
optimizer = []
for i in range(3):
    optimizer.append(torch.optim.SGD(net[i].parameters(), lr = 0.2))
loss_func = torch.nn.MSELoss()

#training neural network
for step in range(300):  #training circles
    for i in range(3):
        output[:, i] = net[i](input[i, :, :])
    prediction = torch.sum(output,1)

    loss = loss_func(prediction,target)
    for i in range(3):
         optimizer[i].zero_grad()
    loss.backward(retain_variables=True)
    for i in range(3):
        optimizer[i].step()

after several circles, all output value become to the same value.
There must something wrong with my code. Could anyone give me some help?
Or are there any better way to implement this network?
Thanks a lot.