Dear experienced friends,
These days I roamed around our PyTorch Forums and tried to find a way to initialize the weight matrix. And I found several ways to achieve that. May I ask which one would you recommend most?

Suppose we have a very simple (but typical) neural network. And our target is to initialize the weight in the first conv1 layer as [[0.,0.,0.],[1.,1.,1.],[2.,2.,2.]]. (As a 3*3 filter)
class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 3)
        self.pool = nn.MaxPool2d(2, 2)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))

        return x


So here are serval ways that we can initialize the weights: (Huge respect to vmirly1, ptrblck, et al.)


Method 1 Define the customize weight matrix inside the __init__:

class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 3, 3)
        self.pool = nn.MaxPool2d(2, 2)

        K = torch.tensor([[0.,0.,0.],[1.,1.,1.],[2.,2.,2.]])  # add the weight here
        K = torch.unsqueeze(torch.unsqueeze(K,0),0) # assign it to the cov1
        self.conv1.weight.data = self.conv1.weight.data * 0 + K

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))

        return x




Method 2: Define the weight after you build the instance (before your training, of course)

net = Net()

# then change the weights outside the class
K = torch.tensor([[0.,0.,0.],[1.,1.,1.],[2.,2.,2.]])  
K = torch.unsqueeze(torch.unsqueeze(K,0),0) 
net.conv1.weight.data = net.conv1[0].weight.data *0 + K 




Method 3 Use the saved-state-dict to update the weights




Method 4 1 Use a class method to achieve that (from tutorials)

class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 3, 3)
        self.pool = nn.MaxPool2d(2, 2)
        self.init_weights()
    
    def init_weights(self):
        K = torch.tensor([[0.,0.,0.],[1.,1.,1.],[2.,2.,2.]])
        K = torch.unsqueeze(torch.unsqueeze(K,0),0)
        self.conv1.weight.data = K

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        return x


I think these are all the available methods that I can find on the Internet. May I ask which one is most recommended? Or any one of them is risky?
Thank you in advance!