Hi,
In certain situations, I would like to use custom weights in certain layers (e.g., from previously trained models), however, I noticed that if I assign weight and bias values manually, they don’t seem to update during training. Below is a simplified example using “normal_” and “zero_”. Here, if I comment out the lines
        self.linear_*.weight.data.normal_(0.0, 0.1)
        self.linear_*.bias.data.zero_()

the model learns. This suggests there’s something from about my approach. Since self.linear_*.weight is already a parameter instance, I thought overwriting *.weight and *.bias would be enough, but it doesn’t seem that way.
Would be nice if someone could shed some light onto this issue!
class Model(torch.nn.Module):

    def __init__(self, num_features, num_classes):
        super(Model, self).__init__()
        
        self.linear_1 = torch.nn.Linear(num_features, num_hidden_1)
        self.linear_1.weight.data.normal_(0.0, 0.1)
        self.linear_1.bias.data.zero_()
        
        self.linear_2 = torch.nn.Linear(num_hidden_1, num_hidden_2)
        self.linear_2.weight.data.normal_(0.0, 0.1)
        self.linear_2.bias.data.zero_()
        
        self.linear_out = torch.nn.Linear(num_hidden_2, num_classes)
        self.linear_out.weight.data.normal_(0.0, 0.1)
        self.linear_out.bias.data.zero_()
        
    def forward(self, x):
        out = self.linear_1(x)
        out = F.relu(out)
        out = self.linear_2(out)
        out = F.relu(out)
        out = self.linear_out(out)
        out = F.softmax(out, dim=1)
        return out

    
torch.manual_seed(0)
model = Model(num_features=num_features,
              num_classes=num_classes)