dpernes:

Doing something like
…
does not work, because param is a copy of the parameters in lstm and not a reference to them.


Did you try? Your snippet works perfectly well for me.
I was surprised at the other thread. The reason you cannot assign the loop variable is because then the loop variable name will just point to something else. You most certainly can modify the element you are looping over and it is exactly that what is sometimes thought of as surprising when you loop over a list of lists and append to the inner variables or somesuch.
In fact, looping over model.parameters() way above is also how the optimizers get the parameters they are optimizing…
Best regards
Thomas