Thank you for the reply.
So is there a way to manually iterate over the parameters of a model and update them without doing the param.data thing, and with ensuring that updates are being tracked? I.e. let torch construct the computational graph that links between the different updates, to do a backward() later on.
I don’t have problems giving up using the built-in optimizers, I just couldn’t find a way to for example iterate over the model as a dictionary (eg. keys-> layer_name, value-> layer_weights), and make keys refer to a different place in memory preserving a track to model’s updates.
If that is doable in torch, thank you for giving me a clue. If not, I would appreciate any suggestions, or alternative torch-based packages that can do that (like JAX with TF).