Solved by DoubtWang in post #2 
              
              
                for i in range(30000):
    optimizer.zero_grad()
    y = (x-2)**2
    y.backward(retain_graph=True)
    optimizer.step()


19000 tensor(1.7238) tensor(1.00000e-02 *7.6316) 
20000 tensor(1.7971) tensor(1.00000e-02 *4.1186) 
21000 tensor(1.8638) tensor(1.00000e-02 *1.8556) 
22000 tensor(1.9210) tensor…