Given some arbitrary model, I would like to manipulate each parameter p before it is involved in a computation. I’m interested in multiplying each p by some learnable constant c. I can’t manually adjust each parameter after the forward pass is completed because then the learnable constant would not be a part of the computation graph. Ideally, the operation p*c would happen before p is used in the forward pass. Then, during the backwards pass, both p and c would receive a gradient wrt the loss.
I looked into hooks to achieve this, but all the posts I have seen so far just use hooks to print out the underlying tensor or gradient information. Is there a way to use hooks in the forward pass to adjust parameters or is this not possible? I could manually define each parameter in the network, and then have control over it; however, my current model has a lot of convolutional layers, so I would prefer not to do this manual work.