I’m trying to train a simple dense network that takes in 6 inputs, and outputs 3. For some reason, regardless of what I put it, it always outputs the same thing right away. Training doesn’t seem to solve it.
Is there something I’m doing wrong here? I’m sure it’s something simple, but it’s driving me a bit nuts.
import torch.nn as nn
import torch

input_size = 6
hidden_size = 64
n_layers = 24
output_size = 3

layers = [nn.Linear(input_size, hidden_size), nn.ReLU()]
for i in range(n_layers):
    layers.append( nn.Linear(hidden_size, hidden_size) )
    layers.append( nn.ReLU() )
layers.append( nn.Linear(hidden_size, output_size) )

net = nn.Sequential(*layers)

x = torch.normal(0,1,(5,6))
print(net(x))

This outputs something like
tensor([[ 0.0090, -0.0353, -0.0797],
        [ 0.0090, -0.0353, -0.0797],
        [ 0.0090, -0.0353, -0.0797],
        [ 0.0090, -0.0353, -0.0797],
        [ 0.0090, -0.0353, -0.0797]], grad_fn=<AddmmBackward>)