I thought that in python x = x - a was the exactly the same as x-=a. But apparently not with tensor :
In the following code the gradient is not set to None after updating the parameter (x-=a case):

import torch
x1 = torch.tensor([2.0], requires_grad=True)
x2 = torch.tensor([3.0], requires_grad=True)
y = (x1 * x2).pow(2)
y.backward()
print(“x1.grad=”,x1.grad)
with torch.no_grad():
x1 -= 0.01 * x1.grad      # <=============================
print(“x1.grad=”,x1.grad)
x1.grad.zero_()
print(“x1.grad=”,x1.grad)

output:

x1.grad= tensor([36.])
x1.grad= tensor([36.])
x1.grad= tensor([0.])

but the following code the gradient is set to None (x = x - a) :

import torch
x1 = torch.tensor([2.0], requires_grad=True)
x2 = torch.tensor([3.0], requires_grad=True)
y = (x1 * x2).pow(2)
y.backward()
print(“x1.grad=”,x1.grad)
with torch.no_grad():
x1 = x1 - 0.01 * x1.grad     # <=============================
print(“x1.grad=”,x1.grad)
x1.grad.zero_()

output:

Traceback (most recent call last):
File “D:/gCloud/GoogleDrive/colabai/notes/torch/grad/zforum.py”, line 14, in 
x1.grad.zero_() # AttributeError: ‘NoneType’ object has no attribute ‘zero_’
AttributeError: ‘NoneType’ object has no attribute ‘zero_’
x1.grad= tensor([36.])
x1.grad= None

Both solutions are ok for me. I just would like to understand this behaviour? Thanks.