Thanks for your reply. If I try to clip the variable after each optimizer step I get the following error:
RuntimeError: Trying to backward through the graph second time, but the buffers have already been freed. Please specify retain_variables=True when calling backward for the first time.
It seems like if you manually mess with the variable state then the variable gets marked dirty or something.
EDIT: Oh, I guess if you only manipulate the .data attribute you don’t get that error. It’s working now, thanks!!