Solved by ptrblck in post #2 
              
              
                Part of your computation graph is outside the loop (xx = x/10). 
After the first optimizer.step() call, the intermediate values will be cleared and thus x cannot be optimized anymore. If you move xx = x/10 into the loop, the code should work.