LIKAN_BLK:

I whant to train variables, used for constraction of this Parameters. So gradient should flow through them.


Does it mean you would like to get the gradient for a and update it?



 LIKAN_BLK:

My aim to get a gradient w.r.t  a  (and deeper). But I can’t replace b in such straighforward way, beacause it already registered as a model parameter.


You could get the gradients for a, if it’s a parameter.
Why do you want to replace self.b, if it’s also a parameter, which is trainable?