Dear @ptrblck
Regarding this topic, kindly I have a follow up question.
Suppose I define the parameters using paramDic like below, then during the backpropagation, Is the error properly propagate from nth parameters toward 1th parameters in the dictionary? or the gradient of the error w.r.t each parameter is computed independently?
What I want to do is to define a tensor parameter for each layer, and during the loss.backward, the gradient of the error w.r.t. these parameters is propagated from the latest parameters toward the lower one.
Thanks
class MyModule(nn.Module):
      def __init__(self):
        super(MyModule, self).__init__()
        self.params = nn.ParameterDict({
                '1': nn.Parameter(...),
                '2': nn.Parameter(...),
                 '3': nn.Parameter(...) ,
                     .
                     .
                     .
                 'n': nn.Parameters(...),
        })

def forward(self, x):
           for paramKey in self.params.keys() 
                 x = self.params[paramKey] * x
        return x