What is the current way to write an SGD-like code without using grad.data, and why is .data not documented anywhere (other than the 0.4 migration guide).
Here is a simple code that solves Ax=b using SGD w/ autograd. I can’t find an elegant way to implement the descent-part w/o using grad.data. Looking at optimizers in the pytorch code base, they all use grad.data.
def update1(x):
    with torch.no_grad():
        tmp = x - 0.0001*x.grad
    x.grad.zero_()
    return tmp.requires_grad_(True)

def update2(x):
    x.data -=  0.0001*x.grad.data 
    x.grad.data.zero_()
    return x

# given A and b, and Ax=b: find x using SGD
A = torch.randn(6, 6, requires_grad=False)
b = torch.randn(6, 1, requires_grad=False)

# choose random x and then search with SGD to find x-hat that is the closest to x in Ax=b
x = torch.randn(6, 1, requires_grad=True)

for i in range(40000):
    loss = torch.norm(A @ x - b)
    loss.backward()
    x = update1(x)
    #x = update2(x)
    if not i % 5000: print(f"{loss: >.6f}    {x}")

update2 is how the core optimizers do it, but if grad.data shouldn’t be used, what’s the idiomatic way to do that? I tried a clumsy workaround in update1 that works, but it looks terrible. And if the idiomatic way is to continue using grad.data, then why is it not documented?
p.s. I understand the potential dangers of using grad.data as explained in the 0.4. migration notes - this is not what I’m asking about.
Thank you!