I found the answer myself. Maybe it will help someone.
After creating a hook it should be removed or it will pile up.
See here 21
So after optimizer.step()
I call h1.remove() where h1 is the returned value of register_hook() function.