Because a gradient update for a leaf Variable a is (with extra terms if you use fancy optimizer) a = a + lambda * a.grad. As you can see, you need the original value of a to be able to perform this update. This is why you cannot change this value inplace, otherwise, you couldn’t do this operation anymore since you would have lost the original a.