Hey Community!
I hope some of you can help me out with this one:
So, I have a network, made of 36 Inputs. Normally, this network has more than half a million data-fields, but currently I´m working with a smaller amount of data. (About 40.000 datas)
The networks aim is to predict the amount of people in a room, depending on their environment. The environment is represented through the 36 input-neurons.
In first place, I tried using 4 layers, with 36 In- and Outputs. The absolute difference between predicted people and the amount of people that should be there, is about 160. This is quite a lot, when the amount only moves between 0 and 2100. When I do this with all the data it is about 200.
So, now I tried to increase the amount of layers and neurons in my network. As you can see in my code, I have 10 layers, all of them have at least 10.000 output-neurons. How can it be, that my network is still that in-accurate? Does the change effect my network in any way? Can someone please tell me? I know the scheme behind neuronal networks, and increasing all these parameters should help…normally…
So this is my Network-Class:
    class Network(nn.Module):
    def __init__(self):
        super(Network, self).__init__()
        self.linear1 = nn.Linear(36, 10000)  #36 Input-Neurons, 36 Output-Neurons, Linearer Layer
        self.linear2 = nn.Linear(36, 10000)
        self.linear3 = nn.Linear(36, 10000)
        self.linear4 = nn.Linear(36, 10000)
        self.linear5 = nn.Linear(36, 20000)
        self.linear6 = nn.Linear(36, 20000)
        self.linear7 = nn.Linear(36, 10000)
        self.linear8 = nn.Linear(36, 10000)
        self.linear9 = nn.Linear(36, 10000)
        self.linear10 = nn.Linear(36, 1)


    def forward(self, x):
        pax_predict = F.torch.relu(self.linear1(x))
        pax_predict = F.torch.relu(self.linear2(x))
        pax_predict = F.torch.relu(self.linear3(x))
        pax_predict = F.torch.relu(self.linear4(x))
        pax_predict = F.torch.relu(self.linear5(x))
        pax_predict = F.torch.relu(self.linear6(x))
        pax_predict = F.torch.relu(self.linear7(x))
        pax_predict = F.torch.relu(self.linear8(x))
        pax_predict = F.torch.relu(self.linear9(x))
        pax_predict = self.linear10(x)
        return pax_predict

    def num_flat_features(self, pax_predict):
        size = pax_predict.size()[1:]
        num = 1
        for i in size:
            num *= i
        return num



network = Network()
print('Netzwerk:')
print(network)
print()

Followed by the training:
optimizer = torch.optim.Adam(network.parameters(), lr=50)
#optimizer = torch.optim.SGD(network.parameters(), lr=0.0022, momentum=0.8)



switch = 1


def training():

    target = y_train_tensor

    for epoch in range(200):

        input = x_train_tensor
        y_prediction = network(input)
        loss = criterion(y_prediction, target)
        wurzel = math.sqrt(loss)
        loss_avg = loss /len(y_train_tensor)
        wurzel_avg = math.sqrt(loss_avg)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        print('Epoch:', epoch, ' Total Loss:', loss.data,'Sqroot:', wurzel)
        print('Average Loss:', loss_avg, 'Avg. Sqrt:', wurzel_avg)
        print()

        plt.scatter(epoch, loss_avg.data, color='r', s=10, marker='.')


    plt.xlabel('Epochen')
    plt.ylabel('Loss')
    plt.savefig('./plot/loss/figure.png', dpi = 300)


    ## Gewichtsausgabe #########################################################################################
    ############################################################################################################


    weights_list = list(network.parameters())
    print('Gewichtsliste:')
    print (weights_list)

    weights_numpy = np.asarray(weights_list)

    print('Array:', weights_numpy)

I hope that someone of you can help me.
Thank you so much in advance!
Kind Regards
Christian Richter