Hi jpj!



 jpj:

I have a fully connected neural network with 78 input features and two hidden layers with 50 nodes each. Output with 15 nodes.
 self.hidden_layer_1 = torch.nn.Linear(78,50)
 self.hidden_layer_2 = torch.nn.Linear(50,50)
 self.output = torch.nn.Linear(50, 15)



This is fine.  At this point self.hidden_layer_1.weight is a tensor with
shape [50, 78].  (The order of the dimensions is correct because of
how pytorch multiplies batch tensors with weight tensors.)


def forward(self, input):
        weights = np.load('./weights.npy')
        weight = torch.from_numpy(weights)
        self.hidden_layer_1.weight = torch.nn.Parameter(weight)



This is your problem,  You are replacing hidden_layer_1.weight, whose
shape is consistent with the number of features in your input batch, with
a different tensor with a shape, [50, 7], that is not consistent with your
input batch.


I’m initializing weights for the first layer and that’s giving me an error. The shape of the weights is (50,7).


Yes.  Presumably the file weights.npy contains a (numpy) tensor of
shape [50, 7].  The question is:  Is this what you want?  The numpy
tensor would be appropriate if your input had 7 features (rather than 78).


What should the shape of my weights be so that I don’t get this error?


If your input has 78 features, hidden_layer_1.weight should have
shape [50, 78] – just as it does before you replace the weight tensor.


I get these weights from running an unsupervised layer which is Restricted Boltzmann Machine.


Why the mismatch in the number of features?  Is your Restricted
Boltzmann Machine processing data with 78 features or 7?
Best.
K. Frank