instead of using for p in layer_data['params'] I worked on the tensor level -> p=layer_data['params'] as u suggested.
  def zero_grad(self):
    for layer_data in self.layers_data_list:
      if layer_data['params'].grad is not None:
        layer_data['params'].grad.zero_()

  def step_sgd(self):
    '''Stochastic gradient descent'''
    for layer_data in self.layers_data_list:
      p=layer_data['params']
      lr=layer_data['lr']
      d_p=p.grad.data
      p.data.add_(-lr,d_p)

it seems that now I’m in overfitting state. Still, I’l be happy if u can explain why I cant iterate over the tensor and run p.grad per weight and instead we do this action on the tensor level :tensor.grad