Which optimizer are you using?
If you are using an optimizer with running estimates, note that even with a zero gradient, the parameters might still be updated, if the running estimates were previously calculated.
E.g. optim.Adam would show such a behavior.
PS: It’s better to post the code directly by wrapping it into three backticks ``` as it makes debugging easier.