The PyTorch way to track trainable parameters is this, take e.g. the Linear module:

  

      github.com
  

  
    pytorch/pytorch/blob/538c30a713a1ee2a3d654c3e1cdf9cc20b7d8c72/torch/nn/modules/linear.py#L85-L100 2



      
      
              self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
              if bias:
                  self.bias = Parameter(torch.empty(out_features, **factory_kwargs))
              else:
                  self.register_parameter('bias', None)
              self.reset_parameters()
          
          
def reset_parameters(self) -> None:
              # Setting a=sqrt(5) in kaiming_uniform is the same as initializing with
              # uniform(-1/sqrt(in_features), 1/sqrt(in_features)). For details, see
              # https://github.com/pytorch/pytorch/issues/57109
              init.kaiming_uniform_(self.weight, a=math.sqrt(5))
              if self.bias is not None:
                  fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)
                  bound = 1 / math.sqrt(fan_in) if fan_in > 0 else 0
                  init.uniform_(self.bias, -bound, bound)
      
    
  


  

  
    
    
  

  


Empty is an uninitialized tensor, then it is initialized (albeit with funny gain parameters, so you don’t want to use these).