If you want a to be a parameter of the model wrap it around nn.Parameter. In your nn.Module add this line: self.a = nn.Parameter(torch.tensor(0.5), requires_grad=True), where 0.5 is the initialization value.