The core reason of “those variables not updated” is because they are not registed in the optimizer. If you look at any optimizer’s constructor, there will be a parameter named params, which is expected to hold the parameters you want to automatically updated by grad.
I’m not sure how you initialized your optimizer. If you do create the optimizer in a most common way, you should make whatever you want to updated automatically as a parameter of the model by using self.register_parameter 21 in the model’s constructor. And then, you could do torch.optim.SGD(model.parameters() since now the model.parameters() will return the variables you just registed.
If you have a hard time to put slop or intercept in the model’s parameters, you could also register those tensors directly to the optimizer by using self.optimizer.add_param_group 9