That might be causing some trouble, as your optimizer won’t be able to update the parameter.
I’ve missed it in the other post, but after updating your weight parameter, you should also re-create the optimizer with the new parameter.
You could probably try to create a new optimizer after each forward call, but that would restrict you to using optimizers without running estimates.