@pytorcher, @SimonW, @yoelshoshan
For my one of the application, I am trying to analyse my m1, m2, m3 parameters values and their gradients. My network architecture is as follows:
class Net(torch.nn.Module):
    def __init__(self, n_feature, n_hidden1, n_hidden2, n_hidden3, n_output):
        super(Net, self).__init__()
        self.hidden1 = torch.nn.Linear(n_feature, n_hidden1)   # hidden layer        
        self.m1      = torch.nn.Parameter(torch.randn(n_hidden1))
        self.hidden2 = torch.nn.Linear(n_hidden1, n_hidden2)   # hidden layer
        self.m2      = torch.nn.Parameter(torch.randn(n_hidden2))

        self.hidden3 = torch.nn.Linear(n_hidden2, n_hidden3)   # hidden layer        
        self.m3      = torch.nn.Parameter(torch.randn(n_hidden3))
        self.predict = torch.nn.Linear(n_hidden3, n_output)   # output layer

    def forward(self, x):
          
        x = F.relu(self.hidden1(x) )
        x = torch.mul(self.m1,x)
        print('My new parameter',self.m1)
        print('gradient',self.m1.grad)   
        
        x =  F.relu(self.hidden2(x))
        x = torch.mul(self.m2,x)

        x =  F.relu(self.hidden3(x))
        x = torch.mul(self.m3,x)
        x = self.predict(x)             # linear output
        return x

However, when I print my parameters using print('parameters',net.parameters), it gives me
<bound method Net.parameters of Net(
  (hidden1): Linear(in_features=784, out_features=10, bias=True)
  (hidden2): Linear(in_features=10, out_features=10, bias=True)
  (hidden3): Linear(in_features=10, out_features=10, bias=True)
  (predict): Linear(in_features=10, out_features=10, bias=True)
)>

m1, m2, m3 parameters have not reflected here. However, values are getting updated. I am trying to understand that how my optimizer getting m1,... parameter when it is not comming in the list. I have defined my optimizer as follows:
optimizer = optim.SGD(net.parameters(), lr=args.lr, momentum=0.95, weight_decay=5e-4)