Thank you @ptrblck and @chenglu. Both of you were right. The parameters were not correctly registered when using a python dictionary.
I tried using nn.ModuleDict and nn.Parameters without success. However, the self.register_parameter() did the trick for me.
I removed the dictionaries for those parameters and added this instead:
self.register_parameter(intercept_name, intercept)
self.register_parameter(slope_name, slope)

To access them in the forward() method I had to do:
for name, param in self.named_parameters():
    if intercept_name == name:
        intercept = param 
    elif slope_name == name:
        slope = param 
       
X = slope * X + intercept 

Now intercept and slope are changing according to the gradient. Is there any way to access these parameters without using the loop over name, param shown above?