Solved by tom in post #3 
              
              
                Hi Stas, 
good to see you around! So drop the .data, wrap in no_grad works here as far as I can see: 
def perfect_update(x):
    with torch.no_grad():
        x -= 0.0001 * x.grad
    x.grad.zero_()
    return x

(I’m having reservations about calling it update when it also zeros grad and about retu…